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Exact Calculation of the Reflection Coefficient 
for Coated Optical Waveguide Devices 


By D. R. KAPLAN* and P. P. DEIMEL* 
(Manuscript received March 8, 1984) 


We derive an exact solution to the problem of the reflection coefficient for 
a coated slab waveguide. A series of computer calculations apply these results 
to a \ = 1.38 wm InGaAsP laser with an active area of 0.2 micron, an active 
area index of refraction of 3.51, and a cladding index of refraction of 3.22. Our 
results show that the correction due to the index step is a few percent for an 
uncoated laser. For antireflection coatings (reflectivity less than 1 percent), 
the correction due to the index step is significant. These results are important 
in choosing coating indices and thicknesses when minimum reflectivities are 
desired (e.g., for a superluminescent diode). The results, calculated over a 
range of indices of refraction of the coating, show that the TE and TM 
reflectivities are minimized at about the same value of the index, 1.84. 
However, the coating thicknesses at which the reflectivity is minimized are 
different for the TE and TM case. For example, when the TE reflectivity is 
minimized, the TM reflectivity exceeds the TE reflectivity by over two orders 
of magnitude. 


I. INTRODUCTION 


The reflectivity of light from optical waveguide devices can be 
modified by the application of suitably chosen coatings. Examples of 
waveguide devices include semiconductor lasers, optical switches, op- 
tical modulators, and optical fibers. In this paper an exact solution to 
the problem of the reflectivity from single transverse mode, two- 
dimensional waveguide devices (such as slab waveguides) is derived 
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and the results are applied to a typical gain-guided semiconductor 
laser. The calculation is only limited to a single transverse mode for 
convenience. The extension to a multimode system is straightforward. 

The problem of reflectivity from an uncoated laser was described in 
detail by McKenna! and his method was employed by Reinhart et al.,” 
Gordon,’ and Krupka.* The eigenstates in the waveguide consist of a 
discrete set of bound modes and a continuous set of unbound modes. 
A numerical calculation for the uncoated laser was performed by 
Ikegami,’ although he neglected the continuous part of the spectrum. 
Pudensi and Ferreira® have developed a method to correct for this by 
combining the continuous modes to form a discrete description of the 
continuum. Kardontchik’ used a method similar to Ikegami to calcu- 
late the reflectivity for three-dimensional laser structures. 

Clarke® suggested an approximate method to calculate the reflectiv- 
ity from an antireflection coated laser. However, that approximation 
is based on truncating the angular spectrum of the incident bound 
state. This approximation is especially poor when the reflectivity is 
calculated for a coating close to the quarter-wave condition. The 
problem of mode conversion by reflection from the mirror facets in an 
uncoated laser was considered in detail by Lewin.? 


Il. MODEL 


The detailed derivation of the reflection coefficient is described in 
the Appendix. This section describes the model for the calculation and 
gives the analytical results. 

The model used for this calculation is shown in Fig. 1. The wave 
incident on the boundary between regions 1 and 2 is described by a 
single bound state, y. The collection of all unbound states, ¢,, plus 
the bound state, y, forms a complete orthogonal set inside the wave- 
guide (region 1). The arrows labeling the states in Fig. 1 denote whether 
the state has a wave vector with a positive z component (e.g., 
J) or negative z component (e.g., 7). 

Inside the coating (region 2) the states may be described as an 
infinite sum of plane waves and evanescent fields. Finally, in air 
(region 3) the complete set is again a set of plane waves and evanescent 
fields. 

The boundary between regions 1 and 2 cannot, in general, be 
described as the boundary between two uniform media. In addition, 
the states, ¢,, are not plane waves. As a result, the Fresnel reflection 
coefficients,!° which were used extensively in previous calculations," - 
48 are not, in principle, valid for this problem. We define An(x) as the 
difference between the refractive index inside the laser at position x, 
(n,(x)), and the refractive index of the cladding layers at position x = 
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Fig. 1—Two-dimensional model used in calculation. 


+oo, (n?). For the case of an asymmetric waveguide An(x) is the 
difference between the refractive index at position x and the average 
of the refractive index of the cladding layers at positions x = +o and 
x = —o (see Section III). In the limit that An(x) approaches zero for 
all x, the unbound states in region 1 again become plane waves and 
the Fresnel reflection coefficients are again valid. In this condition 
the bound state, y, can be expressed as a sum over plane waves.* For 
each plane wave there is a different reflectivity at the interface between 
regions 1 and 2, rj2, where j denotes a particular plane wave trans- 
mitted into region 2. 

The method adopted in this paper is to calculate the reflection 
coefficient, r, in the limit that An(x) approaches zero and then 
calculate an infinite perturbation expansion in terms of a suitably 
chosen integral of An(x). By r we mean the amplitude for the bound 
state to reflect back onto the same bound state. This number is 
important in determining the threshold condition for semiconductor 
laser operation. 

Since we are concerning ourselves with a slab waveguide, we can 
break the problem into the solution of TE modes and TM modes. 


* When An(x) = 0 for all x, there are no bound states. In general, y 
corresponds to a sum of plane waves only at a given surface z = constant. 
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The results for the TE modes are 


oo 


ree = J aaflefrooR) + J aef(o)acey B2*Y, 
where the zeroth approximation to A(s2) is given by 
A (s2) = fe ds,AH(s2, s2)f(s2)(1 — R(s3)), (2) 


and the ith approximation to A(s9) is 


* 7 R(s3) +1 
AM (sz) = A® ah ds3AH(s2, s3)A° (sz) | |. (8 

(sz) (So) he S2 (so, $3) (s3) ee (3) 

The terms of the above expression are now described. The Fourier 
transform of the bound state, f(s2), is given by 


= =o i 7 ik Sox 
f (s2) Tes dx ¥(x) e ; (4) 


where )z is the wavelength of the light inside the coating, Xo is the 
wavelength in vacuum, nz is the index of the coating layer, and 
kg = 2no/do. The complex conjugate of f(s2) is given by f*(se). For 
|s2| <1 in eqs. (1) through (4), se is equivalent to the sin(O), where 0 
is the angle of the transmitted plane wave in the coating relative to 
the normal vector of the surface between regions 1 and 2. Also, R(s2) 
is the Fresnel reflection coefficient for a field incident (at a coating 
angle ©) onto the coated interface.!° Thus, 


T12(S2) + 123(s2) es?) 


hays 
(ss) 1 + ri2(s2)re3(s2) e%°° 


(5) 


where rj is the Fresnel reflection coefficient for the semi-infinite 
boundary between regions 1 and 2, re3 is the Fresnel reflection coeffi- 
cient between regions 2 and 3, and 


ore =n Ts. (6) 
0 


The index of refraction of the coating is nz. The thickness of the 
coating is t, and Xp» is the wavelength in vacuum. 


Furthermore, 
hs) = \/1- (™), (7) 
hi(s,x) = \/1- (2), (8) 
n(x) 
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and 
AH (s., s3) = * ettorst(2-5)) Thi (ss, x)mi(x) — h8(s3)n9], (9) 
0 


where ko = 2x/ do- 

The first term in eq. (1) is the Fresnel approximation to the 
reflection coefficient. The second term in eq. (1) is the correction due 
to the presence of An(x) in region 1. With eq. (3) the higher-order 
correction terms are calculable. These terms are negligible for the 
laser configuration described below. 

Equation (5) shows how the reflection coefficient can easily be 
generalized for a multiple coating condition. The reflection coeffi- 
cients, r12(S2) and ro3(s2), are single-interface Fresnel reflection coef- 
ficients. Only the interface between regions 2 and 3 plays a role in the 
coefficient ro3. For a multiple layer condition r23 is replaced by the 
effective reflection coefficient for a plane wave incident from region 2 
onto a series of layers. 

For the TM, results below a “~” mean that TM parameters (as 
opposed to TE parameters) are used. Thus, 


r™ = Je dso f*(s2)f (s2)R(s2) 


i ds.(1 + R(s2)) [Aen + f ds3AD (so, 0] 
1 ihe =3 

es a 

| 2h2(s0) 2n} 


‘i ds2(1 + R(s2)f(s2)) { ds3AD(s2, $3)f*(s3), (10) 








where 
AD(se, 83) = . i ; dx. eMona(s2-s2)z on — 1} (11) 
AA (59) = | ds3(1 — R(s3))f (s3)Ah(se, 83), (12) 
APO (50) = { ds3(1 + R(s3))f(s2)An(so, 83), (13) 


ioe) 


A4(5:) = AA (5) + i dss 


[-—(1 + R(s3))] 
2h} (s3) 


[1 — R(s3)] 


{a0%09 ob ABO-D(53) Ono false s2), (14) 
1 
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F(s2) 





0 0.200 0.400 0.600 0.800 1.000 1.200 1.400 1.600 1.800 
SINE OF COATING ANGLE, s. 


Fig. 2—The k-space wave functions of TE and TM modes. Solid curve is TE mode 
and dashed curve is TM mode. 


ABO (0) = AB (0) | dsé 


[1 + R(s3)] [1 — R(s3)] 


— ABO-N(35) 


{aap fates s3), (15) 


2h$(s3) 2n} 
and 
Ah(se, 82) = 7 i : dx e'*or2(s2-$2)* hi (so, x) — Af(se)], —«(16) 
and 
An(se, 54) = - i ; dx eMorels2-s2)( n(x) — nf]. (17) 


Ill. NUMERICAL CALCULATIONS 
We now calculate several of the terms given in eqs. (1) through (17). 


3.1 Determination of f(s2) 


The terms, f(s2), are the Fourier transforms of the incident wave 
function, according to eq. (4). The wave function, (x, z), is calculated 
by solving Maxwell equations in the infinite waveguide defined by 
n,(x). The wave functions used in the calculations below are the 
lowest-order TE mode and the lowest-order TM mode. The k-space 
wave functions are shown in Fig. 2. 
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nx) 


ny'{x) 


An‘ (x) 





Fig. 3—Details of index step for an asymmetric waveguide. 


3.2 Determination of An(s2, $2) 


The correction integral, An(s2, sz) in eq. (17), determines the 
strength of the correction term. For a symmetric step waveguide of 
Andngz | sin u ; 27Ne 
—— , where u = 

do u do 

For real waveguide devices the index of refraction of the two cladding 
layers is not, in general, identical. Under these conditions An(x) does 
not approach zero as x approaches +. We must, therefore, show that 
the calculation of An(se, s3) for an asymmetric waveguide leads to a 
finite result. As Fig. 3 shows, n(x) is broken into three terms: n;(x) 
=n, + nj(x) + An’(x). From eq. (8) we obtain 


An(sz, 82) 








thickness d, An(s2, s3) = : (so — $9). 


0 —o 


= ra J dx eftorets-sD nt (x) + i, dx e'tor*22-82)An’(x). (18) 
0 —00 0 ee 


The problem is solved by placing the system in a large box of length 
2b where the length is defined along the x axis. The box then defines 
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the allowed values of k,. By invoking periodic boundary conditions, k, 
is quantized such that 


k, = mbko, (19) 


where 6ky = (27)/b and m is an integer. 
The first integral in expression (18) then becomes 


b Qa 
i—(m—m’)x 
‘ dxe pom n(x) & [cos(2a(m — m’)) ~ 1] = 0. (20) 
ue 


We next allow b to approach infinity. Then, An(s2, s3) is calculable 
with the modified step defined by An’(x). This is a finite result for 
any guiding structure. Note that to solve the problem self-consistently, 
yY must be recalculated for an asymmetric waveguide. In a similar 
manner, Ah(se, s3) [eq. (16)] and AH (se, sz) [eq. (9)] can be calculated 
for an asymmetric waveguide. 


3.3 Calculation of R(sz) and R(s2) 


R and R are calculated in eq. (5) and are used in eqs. (1) and (10). 
Since the integrals extend from —© to +o, R(s2) must be calculated 
for all values of so. There are three critical angles involved, as shown 
in Fig. 4: ; 


Ng 
Sic = 
No 
So = 
0 
ny 
S3¢ = = 6 (21) 
ne 


For sz < sj- an incident wave in region 1 is coupled to a reflected 
wave in region 1, two plane waves in region 2, and a transmitted plane 
wave in region 3. 

For so, < S2 < Sg, an incident wave is coupled to a reflected wave in 
region 1, plane waves in region 2, and an evanescent field in region 3. 

When sa < so < S53. a wave incident in region 1 is coupled to 
evanescent fields in regions 2 and 3 and to a reflected wave in 
region 1. 

When s2 > 83, all the fields are evanescent fields. 

The real and imaginary parts of R(s2) are plotted in Fig. 5a for a 
normalized coating thickness, h = 0, where h = tn2/do. Shown in Fig. 
5b are the real and imaginary parts of the product R(s2)|f|°, which 
correspond to the integrand of the first term in eq. (1). Figures that 
are similar to Figs. 5a and b but are for the case of h = 1/4 are plotted 
in Figs. 6a and b. Note that the terms can be both positive and negative 
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ny n2 n3 


eal 


JRAI< 1 52 < Ste 


|A{=1 j 6 Sic < 52 < S2¢ 


IR] =1 


$a¢ < 52 < S3¢e 


|A|< 1 $2 > S3¢ 





Fig.4—The k-vectors for states involved in the calculation of R(s2) for all 
values of so. 


and that a minimization may correspond to a balancing of the positive 
and negative parts of the integral. The cusps in the real and imaginary 
parts of R(se) occur at the first critical angle. In the region of total 
reflection (from sg = Sa, tO S; = $3,) | R(s2) | is equal to 1. The phase of 
the reflection coefficient varies in this region such that the real and 
imaginary parts of R(s,) take values between —1 and +1. 

When the normalized thickness, h, is ~0.25, the relative weight of 
large so terms (so 2 0.6) is much more significant. Therefore, any 
calculation of rpg or rrm that is limited to the region of small incident 
angles is invalid for antireflection coatings. 


IV. RESULTS 


To demonstrate the above formalism, we have performed reflectivity 
calculations for a \o = 1.3 wm laser with an index of refraction of 3.51 
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UNCOATED 


Re[A(s2)] 


(b) 


UNCOATED 


f2Re[Alsg)] 





0 0.200 0.400 0.600 0.800 1.000 1.200 1.400 1.600 1.800 
SINE OF COATING ANGLE, so 


Fig. 5—Real and imaginary parts of (a) R(s2) for h = 0, and (b) R(s2)|f|? for h = 0. 
f (sz) 1s shown for comparison. 


for the active region and 3.22 for both cladding layers and with an 
active layer thickness of 0.2 micron. We have limited the calculations 
to the zeroth-order correction term. 

The optimum reflectivity, r°?*(n2), is defined as the minimum of the 
r(h) curve for a given coating index ng. In Fig. 7 r°?*(n) is plotted for 
various conditions. In all of these calculations the incident wave 
function is the zeroth mode of the laser waveguide described above. 
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1.75 
1.50 
1.25 
1.00 
0.75 Re[R(so}} 
0.50 


0.25 


-0.25 
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-0.75 





-1.00 
0.35 


0.30 


0.25 f2Im[Als2)] 


f?Re[Al(so)] 


-0.05 


-0.10 


-0.15 





-0.20 
0 0.200 0.400 0.600 0.800 1.000 1.200 1.400 1.600 1.800 


SINE OF COATING ANGLE, sy 


Fig. 6—Real and imaginary parts of (a) R(s2) for h = 1/4, and (b) R(s2)|f|? for h = 
1/4. f (se) is shown for comparison. 


The solid curves show the approximation in which the index of 
refraction of the laser is assumed to be constant and equal to that of 
the active layer. The dashed curves correspond to our calculations in 
which the correction terms are neglected. The dotted curves represent 
the calculation with the zeroth-order correction term included. Note 
that the result that includes the correction term lies between the 
results arrived at by assuming the index of the laser is a constant 
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OPTIMUM REFLECTIVITY, rot 


LASER COATING AIR 


ny = 3.22 
n= 3.51 ng 3 = 1 
ny = 3.22 





1.6 1.7 1.8 1.9 2.0 2.1 2.2 
INDEX OF REFRACTION OF COATING LAYER, ng 


Fig. 7—Optimum reflectivity as a function of index of refraction of coating for (a) 
TE and (b) TM modes for laser structure shown in inset. 


equal to the active area and the result in which the constant index 
is equal to that of the cladding. 

Figure 7 demonstrates the significance of the error obtained when 
approximating the index of refraction inside the laser as a constant 
equal to the index of refraction of the active layer. The results shown 
in Fig. 7 are very similar for both the TE and the TM results. For the 
TE case the reflectivity is minimized at a coating index of refraction 
equal to 1.845. For the TM case the minimum occurs at a coating 
index of refraction equal to 1.840. 

Although both the TE and TM reflectivities are minimized by an 
optimum coating index that is very similar, the coating thicknesses at 
which this minimum occurs are different. This is shown in Fig. 8, 
where both the TE and the TM curves are plotted as a function of the 
coating thickness for a coating index of refraction of 1.840. The 
thickness for which the reflectivity is minimized is greater for the TE 
case than for the TM case. When the TE reflectivity is less than 10~, 
the TM reflectivity is approximately 4 x 107°. 
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TE WITHOUT 


REFLECTIVITY 


TM WITHOUT _ Ma- ~~ CORRECTION 
CORRECTION 


TM WITH TE WITH 


CORRECTION ~ ~~ CORRECTION 





0 0.10 0.20 0.30 0.40 0.50 
NORMALIZED THICKNESS = tna/Ng 


Fig. 8—Reflectivity as a function of normalized thickness for a coating index of 1.840. 
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oO 


PERCENTAGE DEVIATION FROM 
QUARTER-WAVE THICKNESS 
N > 


0 
1.5 1.6 1.7 1.8 1.9 2.0 2.1 
INDEX OF REFRACTION OF COATING LAYER, ng 


Fig. 9—Percentage deviation from quarter-wave condition of optimum normalized 
thickness of coating layer as a function of index of refraction of coating layer. 


From Fig. 8 we learn that to maintain the reflectivity of the TE 
component below 10~+, the film thickness must be controlled to within 
+17A; to maintain the reflectivity below 107° the film thickness must 
be held to +44A. The same conditions for the TM component corre- 
spond to +17A and +53A, respectively. 

In Fig. 9 the percentage deviation of the coating (at the minimum 
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reflectivity) from the quarter-wave thickness is plotted as a function 
of the index of refraction of the coating for the TE and TM cases. 
Because the curves never cross, it will not be possible to simultaneously 
minimize both the reflectivity for the TE and the TM components at 
a level below ~10~*. The TE deviation is approximately 7 percent over 
the range considered. The T'M deviation varies from 2 percent to 1 
percent. 

The calculations can be compared with previous modal reflectivity 
results at h = 0. In agreement with these calculations,® the TE 
reflectivity is higher than the plane-wave result and the TM reflectiv- 
ity is lower than the plane-wave result. 


V. CONCLUSIONS 


We have derived an exact solution to the coated waveguide reflec- 
tivity problem. The solution is valid for an arbitrary index variation, 
ni(x), of the waveguide structure. 

We have observed significant deviations between the exact results 
and the results obtained when the index of the waveguide structure is 
approximated as a constant equal to the active layer index. 

The range of the indices of refraction and the thickness of the 
coating layer to minimize either the TE or the TM reflectivity were 
obtained. The TE and TM reflectivities have their minima at different 
thicknesses. This suggests that tailoring the output polarization of 
light-emitting devices by controlling the coating parameters might be 
possible. 

Also, we have outlined the procedure for calculating the parameters 
to minimize the optical reflectivity. A sample calculation has been 
given for a particular laser structure. Further work is necessary to set 
up general guidelines for coating parameters for the various laser 
structures. 
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APPENDIX A 
TE Reflectivity 


We will use a bra-ket notation, (|), in the following derivation for 
simplicity of notation. In this context a bra-ket denotes integration 
over the x coordinate only. Therefore, the bra-ket depends on the z 
coordinate. Some of the bra-kets involve integrations over products of 
functions defined in adjacent regions. These integrals only have mean- 
ing when evaluated at the boundary between these regions. Equation 
(22) is a summary of some basis states used in the calculation and 
some elementary integrals needed later. To further simplify the nota- 
tion, the system is placed in a large box of unit volume. The integrals 
in the main text of the paper are derived from the sums presented in 
this section by allowing the volume to approach infinity. Note that in 
this section the continuous variable sz has been replaced by the discrete 
variable j. 

The incident wave function in region 1 is 


|v) = e** g(x) (22a) 


|v) = e** g(x). (22b) 


Basis states in region 2 are 


[ee = elt? eis, (k2 + RZ = k3 = (Rone)*), (RE = kz’) (2c) 


d2 
7) == et cite (22d) 
V2 
(FI7’) = a2 5,. (22e) 
(FI) Leo =f (22) 
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(FID) Leo = (FID) Lo = (FIV) Leno = ff (22g) 
IY) le-0 = D fit. (22h) 


In the above expressions g(x) corresponds to the real space modal 
distribution and 6,; is a Kronecker delta function. 

There is a one-to-one correspondence between the wave functions 
j, J and the wave function @,,, where the wave function 7 corresponds 
to a state in region 2 and G,, corresponds to a state in region 3. This 
correspondence is due to the uniformity of the interface between 
regions 2 and 3. Thus we may write 


(F|@n) [=e = OmjOms (23) 


where am is a constant. 
The following calculation consists of demanding continuity of the 
tangential components of the electric and magnetic fields at the two 


interfaces. The wave functions, ¥, represent the electric fields. The 
magnetic fields may be easily obtained by using Maxwell equations. 


We define an operator, h;, which multiplies the plane-wave functions 
in region i by the cosine of the angle between the incident k-vector 
and the surface normal vector. For the TE fields, in which the electric 
field is along the y axis, the magnetic fields can be obtained with 
operator h;: 


Blonte % ni(x)hi| 7) = Fhi(x)ni(x) 17) (24) 
With the help of Snell’s law,’° we write for the operators 

N2S2\" 

h, = 1-(%), So = sin 0, for [s2| <1 
ny 

he =v1l-—- S3 
N2S2\" 

hg = 1- ("22 (25) 
ng 


At the surface between regions 2 and 3 the equation describing the 
continuity of the electric field is written 


J m 
The transverse magnetic field continuity equation is written 


2 [bjngh4| 7) = djnzh}|7)] zs 3 ngh3'Cm|Gm)- (27) 
J m 
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Project with |7) on eqs. (26) and (27) and use eqs. (22) and (28) for 
substitutions to obtain 


b; + d; e774 = Gay; (28) 
ngh} [bj — dj e~"*i] = ngh§cjay;, (29) 
where 
ies Qanghkt 
J No 


and where Xo is the wavelength in vacuum and ¢ is the thickness of 
the coating. 

Equations (28) and (29) are combined to eliminate c;a;, 

(nzh — ngh) 
j (n2h} + nghh ) 
The equation for the continuity of the electric field on the surface of 
regions 1 and 2 is written 


d; = e74) = brby oti, (30) 


|B) + rlO) +S axle) =D GIT) + 417 (31) 


With the use of the identity operators 1 = ¥j|7)(J| and 1 = 
Dsl 77, ea. (31) becomes 


p> lita) + rfp |7') + x on’ 0179) 
= > [by |F") + dy 7"). (82) 


With the help of eq. (24), the magnetic field continuity equation is 
written 


)) [em tenth 179 — GAT — ¥ an(7 | BE mLO17) 
=o nah} [bj |J’) — dy 17’). (83) 


We project on eqs. (32) and (33) with |7) at the surface of regions 
1 and 2 (z = 0). We must consider that since both hi (x) and n(x) 
depend on x, (J | hi (x)mi(x){ 7’) ¥ hi (x)ni(x)(FI7’). 

We obtain 


fi + fir + 2 an(71 os) = b + dj (34) 
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3 (ty ~ for — E on(7'1)) UME m1 
= nyhi(b; — dj). (35) 


We now rewrite the product n,(x)h4(x) as discussed in the main 
text, 


ni(x)hi (x) = nh? + AH;, (36) 


2 
Nos 

m= \/1- (28) 
ny 


and combine eggs. (34) and (35) to obtain 


where 


A; uo 4a. 
filrie nad Ne inte = {1 + re 93 ei], (37) 
1 2 


where r’, and r,, are the plane boundary Fresnel reflection coefficients 
and 


A; = 2, E fs ae x on 19) (7 | AA 17") Leno. (38) 


J’ 


To calculate r we project 1) on eq. (31) at z = 0 and use the fact 
that |) is orthogonal to | ¢,) to obtain 


r= ¥ fib +d — fi). (39) 


If we use eq. (30) to substitute for d;, eq. (37) to substitute for b;, and 
eq. (5) to define R; we obtain eq. (1), the desired result. The last step 
in the calculation requires that the box dimensions approach infinity 
so that the sum tends to an integral. 

The expansion for A; is obtained by equating like terms in eggs. (1) 
and (39), and using eq. (34) to replace bj + d; — f;. We obtain 


[Rj + 1] 
Qn%h? ° 


If (40) is substituted into (38), we obtain eqs. (2) and (8). 





EP 2 an(F| dx) = FR; + A; (40) 


APPENDIX B 
TM Reflectivity 


The TM field is analyzed in close analogy to the TE field. In this 
case the magnetic field is along the y axis. The operator, h;, is useful 
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to determine, for a given angle 9, the tangential component of the 
electric field vector. To obtain the magnetic field, the wave function 
is multiplied by the index of refraction. With these considerations the 
two equations representing continuity at the surface of regions 2 and 
3, after projecting on |7), are written 


na(b; + dje~7#i) = 36a; (41) 
hi,(6; — dje~'#!) = hiéd;. (42) 
Combining eqs. (41) and (42) to eliminate a,¢;, we obtain 


(nsht = nah) 
(nshi, + nzh3) 


~ 


d; = bj 


21d; = bjFo3e4i, (43) 


The conditions for continuity of the electric field and magnetic field 
at the surface between regions 1 and 2 are written 


halv) + harlY) + ¥ hadel de) = he Ls b|7) + ain) (44) 
J 
n(x) iw) +rly + inl’) = Mm b (G17) + iin (45) 
J 
After using the identity operator, projecting on 7), and using the 
fact that at z= 0 (F|m(x)|7’) = (FIm(x) 17") and (7 hA(x)|7") = 
—(F| hi(x)|7’) [see eq. (24)], we get 


hof, — 7h9F; — HEY G(T ba) + Af = 4 (5 — dj) (46) 
k 


and | 
nif — nif +n? D G(F1de) + AP = n2(h +4), (47) 
where 
At = 3 (hr ~ thr - GF’ |x) (FT |hA(x) — ALF") — (48) 
and 


AP = 3 (iy + Phy + Ge(T | An(2) 17"). (49) 


Combine eqs. (46) and (47) by multiplying (46) by n? and (47) by h} 
and then adding them together. Use the fact that 
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n Noh, — mhz 
10 -Se 
Noh, + mhe 


to get 


Af AP 
nt [1 + 7] lap + or a 
Qn» 


eee cee 

Tr ve 

The reflection coefficient is then derived by dividing eq. (45) by 
n,(x) and then projecting on |), so that 














Gib += m3 (7 - i) G+ a). (51) 
F m(x) 
Expand 1/n(x) as 
1 1 1 ng 
aa on oa 7 1 eae) 
= < [1 + AD(x)]. (52b) 
ny 
We then define 
aby = (7) AE 1] (52c) 
is aac aa : 








and combine egs. (51) with (52) to obtain 
etal .2 is se ga a dae 
Lhifit Fea 7 » is 12, aby it] bj(1 + Fose**i). (53) 
Ps J J 


Now substitute (50) for b; to yield an expression for the reflection 
coefficient: 


2 . AS. AB 
F=VfFhR + 2D fF (L+ Aili +3 aby i#]| 3% + 2] 
j j 1 


+32 aby7t] 0+ Ri. 68 
J J 
We must now form a series expansion for A¢ and A?. 
Combining eqs. (47) and (48) and (50) we eee 
*B 


~X ns. Peg a4 A 
hy + D a(F1d.) = Bf + 01+ Bl oe oi “1 - Rg 5. (5) 


Equation (55) is now plugged into (48) and (49) to obtain eqs. (12) 
through (15). 
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The LT-1 Connector Family of Transmultiplexers 
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The LT-1 family of transmultiplexers was designed to provide a flexible 
and economical means of interconnecting the widely deployed long-haul analog 
transmission plant with the emerging network of digital switches and facilities. 
The LT-1 connector eliminates redundant interfaces and uses custom inte- 
grated circuits. The most significant improvement is in the analog signaling 
circuits—the function of an entire circuit pack is reduced to circuitry occupying 
less than 2 square inches. Reuse of circuits designed for other systems under 
development helped to introduce this technology rapidly. The original design, 
targeted for the 4ESS™ market, was enhanced with three additional designs, 
which expanded the signaling capability to include special services, added a 
digital echo canceler for satellite applications, and provided compatibility with 
international signaling. 


I. INTRODUCTION 


In 1976 the first digital toll switching office 4ESS* switch was 
placed into service in the Bell System and plans were finalized to 
develop the LT-1 connector. It was clear that in the decade ahead, 
most of the vast network of analog toll trunks would need a digital 
signal format at the switching system interface. This was the primary 
motivation for developing the LT-1 connector—to economically ter- 
minate analog trunks on a time division, digital switching machine. 
This equipment is referred to generically as a transmultiplexer. 


* AT&T Bell Laboratories. 
t Trademark of AT&T Technologies, Inc. 
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Il. LONG-HAUL TOLL NETWORK 


Plans for deploying the LT-1 connector were closely linked to the 
4ESS cutover program. At the outset it was clear that even with 
expedited development and manufacture, initial shipment would not 
be possible before the year 1979; 4ESS installation schedules indicated 
that actual service on the equipment could be achieved in 1980. In the 
5 years (1979 through 1984), plans called for retiring about 120 
electromechanical 4A toll switches, and replacing them with about 90 
digital 4ESS switches. Figure 1 illustrates an estimate of analog trunk 
needs in the LT-1 connector availability time frame. This estimate 
indicated a program to deploy 900 frames of LT-1 connectors providing 
432,000 analog trunk terminations each year. By 1985 all but 14 
percent of these analog trunks will be switched by digital 4ESS 
machines; over 83 percent of the analog trunks will be terminated 
with transmultiplexer equipment. 

The intertoll transmission facilities are predominately analog and 
include coaxial cable, FM radio, and, most recently, single-sideband 
AM radio. This situation is not expected to change until the advent 
of long-haul optical fiber and digital radio transmission systems in the 
late 1980's. 

Toll-connect transmission facilities also incorporate significant 
amounts of analog, as well as digital, equipment. The division is about 
equal; the current, modest analog majority is expected to change to a 
modest digital majority by the mid-1980’s. 


ALL ANALOG TOLL TRUNKS ~~ 


NUMBER OF TOLL TRUNKS IN THOUSANDS 





1979 1980 1981 1982 1983 1984 1985 
CALENDAR YEAR 


Fig. 1—Long-haul, toll network, analog trunk estimate. 
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Il. OTHER APPLICATIONS 


Another application of the LT-1 connector is the more general 
interconnection of digital, T-carrier transmission facilities with ana- 
log, L-type facilities. When this potential a” -lication was surveyed, 
the telephone operating companies indicatea._ .irly limited needs (es- 
timated to be less than 10 percent of 4ESS switch analog trunk 
termination applications). It was thus decided to defer the facility 
interconnect development (LT-1B) and concentrate on making the 
4K&SS switch version available as soon as possikie. Also, the need to 
accommodate many signaling types in the facility interconnect appli- 
cations made possible a less costly version, tailored to the less stringent 
signaling requirements of the 4ESS switch application. 

The digital access and cross-connect system (DACS) has consider- 
ably changed the facility interconnect application. DACS “grooms” 
digital facilities by collecting individual special-service channels from 
a number of different digital systems and routing them to the special 
office equipment required to operate and maintain them. Similarly, 
the individual channels can be distributed to many different digital 
systems to reach the end customers. The LT-1B facility connector, in 
combination with DACS, will collect and distribute special-service 
channels, utilizing analog transmission facilities in a similar manner. 


IV. DESIGN PHILOSOPHY 


Two design techniques were originally investigated. The first ap- 
proach was to merge the block diagrams of the individual systems— 
frequency division multiplexing (FDM), single frequency signaling, 
and digital pulse code modulation (PCM)—into a single diagram. 
Immediately, it was clear that much of the interface circuitry in an 
integrated terminal was redundant, consisting primarily of magnetic 
components used to isolate the circuit packs from the outside world. 
Also, standard interfaces for signaling and transmission were unnec- 
essary for this application. 

Once these standard interfaces and the interface circuitry were 
removed, up-to-date integrated circuit techniques would be applied to 
the remaining circuits. The result was a compact modern design that 
was within the bounds of existing manufacturing technology. Figure 2 
shows the block diagram. 

The 12-channel analog FDM signal is buffered and distributed on a 
low-impedance bus to the 12 channel units. On the channel unit, the 
4-kHz band of interest is selected by a crystal bandpass filter and 
demodulated to voice frequency (VF). Any signaling information pres- 
ent is detected and sent to the digital transmit unit in the form of a 
digital logic signal. The voice signal is then sampled at 8 kHz, and the 
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Fig. 2—LT-1 connector. 
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Fig. 3—All-digital transmultiplexer. 


GROUP 2 


samples are time-division-multiplexed on a pulse amplitude modula- 
tion (PAM) bus with 23 other channels. These samples are encoded, 
along with the signaling information, into the standard 24-channel 
DS-1 signal. The other direction of transmission inverts this process. 

Also, some of the circuits are identical to those used in two recently 
completed developments—D4 and N4. This reduces manufacturing 
needs and training requirements for the operating companies. 

The second approach considered was an all-digital design. Very 
simply, it would convert the analog group signal into a digitized version 
with a high-precision, high-speed, analog-to-digital (A/D) converter 
and perform the demodulation, signal detection, and time division 
multiplexing functions entirely with digital hardware. It could time- 
share the circuitry over a large number of channels and thus achieve 
the well-known benefits of digital circuitry. Figure 3 shows a simplified 
block diagram of the terminal. 

Selecting the approach for final development required an estimate 
of the cost, size, and power consumption of each design, as well as an 
estimate of the development time and risk. The results of the study 
with the per-channel parameters normalized to one are shown in Table 
I. To the authors’ knowledge, this is the first time in which such a 
study has shown that an all-digital version of what is essentially an 
analog transmission function could be cost-competitive with a modern 
per-channel design. We believe this is caused by two factors. The first 
factor is the degree of complexity possible with very large-scale inte- 
grated (VLSI) devices; tens of thousands of switching elements can be 
integrated into a single device. The second reason is inherent in the 
transmultiplexer itself in that an A/D conversion need be performed 
only in one direction of transmission. The fact that the cost estimate 
for an all-digital transmultiplexer was comparable to the per-channel 
design suggests that continued development in this area is warranted, 
and further, it seems likely that an all-digital design soon will become 
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Table |[—Transmultiplexer 
parameter Comparison 


Per Channel Digital 


Cost 1.0 1.0 
Size 1.0 0.5 
Power 1.0 2.0 


more economical than the per-channel design. However, the all-digital 
design was considered to have significantly greater risk and a longer 
design interval than the per-channel approach. The cutover of a 4ESS 
office is planned several years in advance, and the unavailability of a 
significant amount of terminal equipment could jeopardize the service 
date. This would result in a large amount of capital not being used 
productively. Therefore, it was decided to proceed with the per-channel 
transmultiplexer for final development and, at the same time, con- 
tinue, on an exploratory basis, the all-digital transmultiplexer. 

As we stated earlier, LT-1 initially was used to terminate analog 
trunks on the 4ESS switch. In the beginning, two types of circuit 
designs were made available—the common channel interoffice signal- 
ing (CCIS) channel unit, and E and M channel unit. Both channel 
units share identical transmission circuits. The difference between 
them is that the E and M channel unit has additional circuits to detect 
the 2600-Hz signaling tone and map it into the robbed eighth-bit 
signaling format used in the DS-1 signal. 


V. CHANNEL-UNIT DESCRIPTION 


Figure 4 shows that the channel unit performs three separate 
functions: analog modulation, signal conversion, and digital sampling. 
On the analog side, the channel unit must interface with a broadband 
FDM signal that contains twelve voice channels in a basic group. 
These must be converted to voice frequency. Signaling information, if 
present, must then be extracted and the VF signal must be sampled 
and applied to the PAM bus to interface with the digital common 
circuits. These three functions are described in the following para- 
graphs. 


VI. ANALOG MODULATOR 


A single custom integrated circuit performed the analog modulation 
functions. This circuit had been thoroughly characterized previously 
when it was used in LMX-3 and N4. In this circuit the signal voltage 
is converted into a current and applied to a pair of emitter-coupled 
pairs. These are switched on and off under the control of the carrier. 
The switching effectively multiplies the signal by plus and minus one 
and performs the modulation process. Because of the balanced nature 
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Fig. 4—LT-1 channel unit. 


of the circuit, the VF signal and the 2600-Hz signal are effectively 
summed at the same time. The double-sideband output of the modu- 
lation process is filtered with a discrete crystal filter, which selects the 
lower sideband. 

The reverse of this process occurs in the opposite direction. The 
broadband analog signal is applied to a crystal filter, which selects one 
channel, and the modulator circuit converts it to VF. Equalization to 
compensate for low frequency rolloff is provided, as well as a variable 
gain adjustment to compensate for amplitude misalignment in the 
analog facility. 


Vil. DIGITAL SAMPLER 


The next step in the process is to convert the VF signal into PAM 
samples. This is accomplished with a standard D4 high-pass active 
filter and junction field-effect transistor (JFET) sampler. The sampler 
timing is controlled by a channel counter that interleaves the samples 
from 23 other channel units to create the complete PAM bus signal. 

In the reverse direction, a second JFET gates in the appropriate 
sample from the receive PAM bus, also under the control of the 
channel counter. This sample is then filtered in a low-pass filter to 
reconstruct it into a continuous waveform and applied to the modu- 
lator. These circuits were designed to be compatible with the common 
circuits used in D4 so that the latter could be used interchangeably in 
an office that has both LT-1 and D4. 


VII. SIGNALING 


Figure 5 shows a block diagram of the functions in the signaling 
receiver. The requirement is to distinguish between the presence of 
the true 2600-Hz signaling tone and talker energy, which may have 
frequency components around 2600 Hz. To accomplish this, the signal 
is processed by two active filters. Since one has a transmission peak 
at 2600 Hz (SIG) and one has a null (GUARD), the relative amount 
of energy from the two filters determines if the input is a pure 2600- 
Hz signal or if it contains significant energy at other frequencies. 
Because the circuit has short time constants so it can respond to pulse 
inputs in a few milliseconds, the output of the comparator, EN, must 
be further processed by pulse-correcting circuits before being trans- 
mitted to the digital circuitry. 

During certain high noise conditions, such as those that occur during 
radio fades, the amount of energy detected by the guard amplifier can 
simulate talker conditions. If no other action was taken, it would 
appear that a trunk seizure had occurred. Since this would happen on 
all trunks simultaneously, the switching machine could become over- 
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Fig. 5—-Signal receiver. 


loaded. To prevent this, the gain of the guard amplifier is reduced 
after 185 ms of being in the idle state. 

Figure 6 shows a simplified block diagram of the signal transmitter. 
When the 2600-Hz signal is first transmitted, it is applied at a —8 
dBm0 level, the high level. Because the continuous application of that 
much power in every channel exceeds the power level that the analog 
facility was designed to carry, the power is reduced to —20 dBm0 after 
400 ms. In Fig. 6, the buffers convert the unregulated transistor- 
transistor logic (TTL) signals 2600H (high) and 2600HL (high-low) 
into controlled amplitude square waves. Generating none, one, or two 
of these signals and summing them produces tone off, low tone, or 
high tone. This output is added to the VF signal and modulated to the 
group band. 

The balance of the signaling circuitry, which consists of seven 
timers, is shown in Fig. 7. The timers perform pulse correction in both 
directions, M lead correct and E lead correct; control the amplitude of 
the tone (HI-LO); enable an active filter to remove the received tone 
(BAND ELIMINATION FILTER TIMER); control the gain of the 
guard amplifier (G TIMER); and disable the transmission path during 
signaling activity (CUT A, CUT B). These seven timers are all 
implemented on two custom digital integrated circuits. 

Because the LT-1 architecture does not require standard interface 
points for VF and signaling, the entire signaling function can be 
accomplished with these three chips mounted on a hybrid integrated 
circuit, along with an active filter and a few discrete components. 
Before LT-1 was developed, signaling required an entire printed wiring 
board. Further, since the timers are all digital, no problems exist with 
aging or component drift. 
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These two types of channel units are sufficient for all 4ESS switch 
trunks since only E and M, or CCIS signaling is used in that machine. 

After the CCIS and E and M channel units were introduced, a 
different CCIS channel unit equipped with an echo canceler was 
developed. Echo canceling is implemented by a VLSI chip’ that can 
generate an approximation of the echo introduced at the 2- to 4-wire 
points in the transmission path. The echo canceler was first used in 
LT-1 on satellite circuits. As is well known, the subjective annoyance 
of echo increases with delay. The round-trip delay through a geosta- 
tionary satellite is approximately 600 ms. Long-distance connections 
whose echo performance would have been satisfactory on terrestrial 
circuits became unsatisfactory on a satellite. Further, echo suppres- 
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sors, traditionally used on long terrestrial circuits, are unsatisfactory 
on domestic satellite circuits. The echo canceler has been shown to 
provide a grade of service on a satellite that is equivalent to terrestrial 
circuits. 

LT-1 is a natural point in the transmission path to place an echo 
canceler because it is located at the termination of an analog trunk 
onthe switching machine. Further, since the VLSI echo canceler 
was implemented as a digital device, the canceler will interface with 
the digital signals already present in LT-1. Figure 8 shows the initial 
LT-1 transmission path. The receive unit converts the PCM to PAM 
and distributes it to the channel unit. Similarly, the transmit unit 
converts the PAM samples from the channel unit to PCM. Figure 9 
shows how the echo-canceler unit operates. In the digital transmitting 
direction, the same PAM-to-PCM conversion occurs, but the PCM is 
sent back to the channel unit for echo canceling. In the receiving 
direction, the digital-to-analog (D/A) converter is not used. Rather, 
the received PCM is sent directly to the echo canceler as a digital 
signal. Having access to both directions of transmission allows the 
canceler to make an approximation to the echo path and thereby 
cancel the echo. The output of the canceler is a digital signal, which 
must be converted to VF by a per-channel decoder. The analog portion 
of the terminal is unchanged. 


IX. COMMON UNITS 


The line interface, transmit, and receive units are unchanged D4 
units. The alarm control units (there are two versions) use the D4 
alarm control unit layout but with features not needed for LT-1 deleted 
to reduce their cost. The version used depends upon whether LT-1 is 
located in the same office as the 4K SS switch or in another office with 
connection to the 4ESS switch provided by a T1 line. 

The D4 line build-out circuit was incorporated into the LT-1 digital 
access units, of which there are two versions. These units provided 
the access points for testing and aligning the digital access time-slot 
selector (DATS). One of the versions provides a looping point, so that 
the 4ESS switch could be looped back on itself prior to LT-1 being 
put into service. Which digital access unit is selected depends upon 
whether the 4ESS switch is to be looped at the LT-1 or at a DSX-1 
cross-connect frame located in the circuit between the LT-1 and the 
4ESS switch. 

The combine-and-split unit provides low-impedance summing and 
driving points for the 12 channel filters. Finally, a power unit provides 
+12V and +5V power to all plug-ins in a double digroup. With the 
development of the echo canceler, an echo-canceler timing unit has 
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been added. Also, a new power unit, with additional +5V power 
capability needed to drive the echo cancelers, has been designed. It is 
interchangeable with the original unit. 

These various units, together with the equipment shelves, are the 
major components of the LT-1 double bay frame. 


X. MAINTENANCE CONSIDERATIONS 


An integral part of the design for the 4ESS switch was a plan for 
installing and maintaining the LT-1. Existing equipments used the 
switched maintenance access system (SMAS) to enable VF testing to 
be done on various trunks from a remote test position. Since LT-1 has 
no VF access, a new testing method was needed. DATS filled this 
need.” The DATS is essentially a portable D4 channel bank under 
microprocessor control. It can address any two of the 24 channels (one 
for use as an order wire, the other for test) and measure the VF power 
or frequency or the state of the signaling bits. It can generate a set of 
tones at three frequencies and four levels from information stored in 
a read-only memory (ROM). In addition, it can interface with any VF 
equipment and digitally encode external test signals or convert the 
digital signal on any channel into the equivalent VF signal so that it 
can be measured by using standard test equipment. This capability 
gives the DATS several advantages. First, since it is measuring and 
generating digital information, there is no error in the measurements 
caused by circuit misalignment beyond the measurement point. Sec- 
ond, since the DATS is microprocessor-controlled, macro programs 
that very quickly do repetitive tasks characteristic of trunk lineup 
procedures can be, and indeed have been, written. 


XI. FACILITY CONNECTOR 


After the LT-1 design for 4ESS switch applications was completed, 
the more general problem of a facility connector, to be designated LT- 
1B, was investigated. Here the purpose was to connect two 12-channel 
group signals to a 24-channel digital signal independent of trunk type 
or orientation. The differences in requirements were primarily related 
to signaling. The types of signaling units currently in production were 
studied to find any commonality in the way they represent trunk 
conditions in the analog and digital signaling domains. The results 
indicated that a great deal of commonality exists, while the differences 
between the various units are typically in the interface circuitry, e.g., 
two wire or four wire, rather than in the mapping of 2600 Hz into the 
A and B signaling bits. It follows that for LT-1B, a single design could 
handle all of the 2-state signaling. The 3-state design is more complex. 
Here a 20-Hz modulation is placed on the 2600-Hz tone to indicate 
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the third state. A channel unit must be able to detect or generate the 
modulation, depending upon the orientation of the analog and digital 
sides of the transmultiplexer relative to the switching office. Finally, 
a fourth type of channel unit without a signaling circuit was developed 
for CCIS applications. 

With 4 signaling types and 12 channel filters, it was obvious that 
the signaling circuits should be placed on a small module that could 
be plugged into the main channel unit. This would give complete 
flexibility to the operating company planners and simplify sparing. 

In addition to the enhanced signaling capability, LT-1B provides 
features not required in LT-1. 

1. The channel counting sequence in each digroup can be made 
compatible with D1D, D2, or D3/D4 channel banks in the field. 

2. The transmitting clock can be locked to either incoming digroup 
or it can run free. 

3. The digital side can operate in mode 2 (DS-1C) or mode 3 (two 
DS-1s). 

4, An optional 2-way carrier failure alarm (CFA) with trunk proc- 

essing can be provided. 
This last feature is necessary to allow arbitrary routing of the analog 
groups. Consider the arrangement shown in Fig. 10. A short-haul, 24- 
channel digital DS-1 signal from Lawrence, MA is converted, by an 
LT-1B in Boston, into two long-haul analog groups that terminate in 
different cities, Cleveland and Chicago. If a failure occurs on the 
digital digroup, the normal red and yellow alarms will be propagated 
and trunk processing will occur at the Lawrence, MA end. The LT- 
1B will remove the outgoing CFA pilot, and the channel banks in 
Cleveland and Chicago will be processed. However, if a failure occurs 
on one of the analog trunk groups, it cannot be propagated to the 
digital banks without all 24 trunks being taken out of service. There- 
fore, supervisory trunk processing is performed by the LT-1B only on 
the 12 trunks affected by the failure. This allows service to continue 
on the other group. 

To accommodate the various kinds of signaling, three trunk proc- 
essing options may be selected on a per-channel basis: 

1. On-hook for 2.5s followed by off-hook 

2. On-hook 

3. Off-hook. 

The purpose of these three options is to terminate billing when a 
failure occurs on any calls in progress and to prevent false ringing or 
other similar problems on special-service lines. 

A particularly important application for LT-1B is its use, in con- 
junction with DACS, to groom analog facilities. The DACS, which was 
introduced in 1981, performs a time-slot interchange function, similar 
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to that of the 4ESS switch. Its role in the network is to separate 
switched message and special services on T1 lines. The combination 
of LT-1B and DACS will allow the same type of network flexibility in 
the analog plant. 


XII. THE LT-1E CONNECTOR 


Another variation in the basic LT-1 design is compatible with 
International Telegraph and Telephone Consultative Committee 
(CCITT) transmission and signaling on the analog side and the 
standard North American DS-1 on the digital side. 

The LT-1E connector is the CCITT version of LT-1. It required 
new carrier supply units, a new channel unit, and a new combine and 
split unit. The channel unit was a combination of CCITT analog 
modem and D4 circuits. The analog circuit used 2-step modulation 
requiring both 12-channel carriers in the 192- and 236-kHz band and 
a 128-kHz premodulation carrier. The advantage was that only one 
code of channel filter was required, and hence, the same code of 
channel unit could be used for all 12 channels. In addition, the channel 
unit used out-of-band signaling at 3825 Hz. This signal tone could be 
transmitted at either a high level of —5 dBm0 or a low level of —20 
dBm0, and it could be used as “tone on” or “tone off” in the idle (on- 
hook) state. 

The new combine-and-split unit provided CCITT levels at the group 
distributing frame (GDF). It could insert a pilot frequency of either 
84.08 or 104.08 kHz into the analog path and included regulating 
circuits for the receive pilot. 

The carrier supply used the LT-1 shelf but was made up of 12 
carrier generators whose frequencies were equally spaced between 192 
and 236 kHz, a premodulation generator, a pilot generator, a signal 
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generator, and a power unit. The last four items were duplicated for 
reliability. 


XIII. LT-1 EQUIPMENT DESIGN 
13.1 Introduction 


The similarities between the LT-1 carrier supply needs and the 
needs of N4 have been pointed out earlier. Secondly, a number of 
digital D4 common circuits could be used directly in LT-1. Finally, 
there was also a similarity in systems’ architecture. In its simplest 
form, LT-1 converted and combined two sets of 12 separate analog 
signals to form a (digital) DS-1 signal. In D4, 24 separate analog 
signals were combined to form a DS-1 signal. Additionally, it would 
be of economic benefit if components, piece parts, or entire units from 
N4 or D4 that were developed ahead of LT-1 could be used in LT-1. 

The plan was to use a reduced N4 carrier supply, the D4 channel 
bank shelves, and many of the D4 common units, with the LT-1 
channel units and certain of the common units being new. 


13.2 Equipment considerations 


LT-1 operates in conjunction with the digital interface frame (DIF) 
to terminate analog signals on the 4ESS switch. Five DS-1 signals 
(120 channels) from the LT-1 are combined in the DIF to form a DS- 
120 signal, which is the input to a 4ESS switch. To maintain compat- 
ibility with the 4ESS switch, LT-1 was made up in multiples of 120 
channels. 

The decision to use N4 and D4 equipment shelves meant using an 
unequal flange bay with a nominal width of 2 ft 2 in. Further, since 
LT-1 was to be installed in a 4ESS switch, the bay height (7 ft) was 
set by the new equipment building standards (NEBS), to which both 
the DIF and the 4ESS switch conformed. An objective of 480 channels 
for LT-1 in two 7-ft bays was established. This was accomplished by 
a special channel unit design described below. 


713.3 Channel units 


The basic D4 shelf assembly consists of four shelves, each with 12 
channel units. The shelves are die-cast type with 60 equally spaced 
slots on both sides for the printed wiring boards. Each channel unit 
requires four slots, with the remaining ones left over for the common 
units. This arrangement, if used for LT-1, would result in empty shelf 
space since not all the D4 common units are needed. To alleviate this 
problem, the channel units were rearranged so that 48 channel units 
would fill two shelves (that is, 2-1/2 slots per channel), with the third 
one for the common units. This was done by turning every second 
channel unit assembly over so that within a pair of assemblies the 
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boards were on the outside and the components extended towards the 
middle. The components on each board were also rearranged so that 
the high and low components mesh properly without touching. Each 
double-channel unit thus occupies five slots. 


13.4 The LT-1 double digroup 


Since LT-1 requires only three of the four shelves, the D4 shelf 
assembly was reworked. New castings slotted on one side only were 
designed to be used either as the top or the bottom and were attached 
by side panels to create a 13-1/2 in. high, 3-shelf module. The center 
shelf accepts the D4 common units, and the top and the bottom 
shelves contain the LT-1 channel units. Each shelf assembly is thus a 
48-channel, self-contained module; a 480-channel system would re- 
quire two 7-ft bays. 

A fully equipped system (480 channels) with EK and M channel units 
requires 35 different codes, or circuit packs. Of these, six are double 
channel units, ten are new to LT-1, and the remainder are taken 
directly or with slight modifications from N4 or D4. 


13.5 Carrier supply shelves 


Since the LT-1 and N4 were progressing in parallel, the N4 supply 
shelf was redesigned to meet both LT-1 and N4 carrier supply require- 
ments. The new castings are slotted on one side only so that they can 
be used as the top or the bottom of a shelf. These castings were screwed 
to the side plates and to a connector panel (backplane), with provision 
for 24 carrier units. 


13.6 Carrier supply units 


The 24 carrier supply units, including those for the carrier failure 
alarm, were made from 20 codes. Twelve carrier generators provide 
separate carrier frequencies, of which six were used unchanged from 
N4, while the other six were modified to remove a transmitted carrier 
circuit not needed in LT-1. The twelve received their reference fre- 
quency from a 4-kHz generator driven from the office primary fre- 
quency supply. This unit also provided 4 kHz to the 100.08-kHz carrier 
failure alarm generator. The 2600-Hz signaling frequency was gener- 
ated by a free-running oscillator. These three units were duplicated 
for reliability. Finally, there was an alarm unit and duplicate power 
units. 


13.7 Channel units 


A double channel unit consists of two printed wiring boards with 
components between them. The terminal (pin out) arrangement was 
kept the same on both boards so that all channel units had the same 
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functions on the same terminals. This simplified bus wiring on the 
connector panel backplane for all common functions. Similarly, com- 
ponents that would be adjusted by craft personnel were kept in the 
same relative position on faceplates. Except for layout, the only 
difference among 12 channel units of the same general type, within a 
group, was the frequency of their filters. 

The same pair of printed wiring board layouts was used for both E 
and M and CCIS double-channel units, with the E and M signaling 
circuits left off the CCIS units. However, a switch was provided on 
the E and M unit so that it could be converted directly to CCIS. 

Later, when the echo-canceler circuits were incorporated into the 
channel units using a single, 2-1/2 position unit, the high components 
were placed at the top of the board and the low components at the 
bottom. This was reversed on the adjacent board, and by using notched 
faceplates that conformed to the component arrangement, one unit of 
an opposing pair slid past the other as it was moved in and out of a 
shelf. . 


13.8 LT-1 double bay frame 


The LT-1 connector is housed in a double bay frame made from two 
standard 7-ft unequal flange bays set side by side. The base is 4 ft 4 
in. wide and 12 in. deep. 

Prior to assembly, the individual equipment shelves are wired. The 
transmit and receive leads from the digroups are then connected to 
the connectors on the bays. 


13.9 Cabling 


The transmission path for LT-1 is connected to the GDF on the 
analog side, and to either the DIF, the digroup terminal (DT), or the 
digital cross-connect frame (DSX-1) on the digital side. 

The analog connections are made using twisted shielded pairs (761A 
type), with the shields grounded at the GDF and floating at the LT-1. 
There are two pairs—one transmit and one receive—for each group. 
They are terminated on the combine-and-split unit connector at the 
back of the double digroup. 

The digital connections are made using 600B (type) cables. The 
cables for the DIF and DT connections are fully connectorized, while 
the DSX-1 cable is connectorized at the LT-1 end and wire wrapped 
at the DSX-1. The aluminum cable sheaths are grounded at the LT-1 
bay, and there are separate cables for the transmit and receive paths. 


713.10 The LT-1B connector 


The LT-1B is packaged in three versions. The first is a 7-ft unequal 
flanged bay, which, from the top, contained a fuse and alarm panel 
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Fig. 11—LT-1B connector primary and secondary bays. 


designed for LT-1B, the LT-1 carrier supply, and five modified double 
digroups. The double digroups have two more connectors on the 
common shelf connector panel so that they can be used for LT-1 and 
LT-1B with and without T1C operation. New channel units and a 
synchronizer/desynchronizer (SYNDES) unit (a D4 circuit) were 
added. This was the primary bay of a pair and contained 240 channels. 

The second bay holds an additional 240 channels. It contains five 
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double digroups and an optional 660 communications panel located 
between the third and fourth digroups from the bottom. This bay 
receives its power and carriers from the primary bay and was alarmed 
through the bay. The LT-1B primary and secondary bays are shown 
in Fig. 11. 

The third bay is 11 ft 6 in. high and self-contained, with a total of 
eight double digroups (384 channels). Its layout is similar to the 7-ft 
primary bay except that between the third and fourth double digroups 
there is a communications panel, and above the fuse and alarm panel 
there are three more double digroups. 

The channel units are similar in concept to the LT-1 echo cancelers 
but include four removable circuit modules (daughter boards) contain- 
ing the signaling circuits. They are completely interchangeable and 
can be mounted on any of the 12 channel units. 


XIV. SUMMARY 


This article has described the LT-1 connector design. The original 
design, intended for 4ESS applications, would accommodate E and M 
and CCIS signaling only. It took maximum advantage of the common 
technology of D4 and N4, as well as custom integrated circuits. 
Additional features and signaling capabilities were added to the design 
for facility interconnect applications. An echo-canceler plug-in was 
added to improve the grade of service on satellite circuits. Finally, 
technology developed for international markets was incorporated into 
the design to interface with an export version of the 4ESS switch. 

The LT-1 design has been very flexible in meeting the needs of 
AT&T Communications and the Bell Operating Companies. It eco- 
nomically converts between analog and digital transmission, a critical 
function as AT&T Communications and the Bell Operating Compa- 
nies move toward an all-digital network. 
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Electrical Transmission Lines as Models for 
Soliton Propagation in Materials: Elementary 
Aspects of Video Solitons 
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Distributed electrical and mechanical transmission lines are useful models 
for nonlinear wave motion with dispersion in many interesting physical 
systems. Nonlinear wave motion with dispersion produces solitons in optical 
fibers. A soliton will propagate along an appropriate transmission line with 
constant velocity and without change in shape. The shape remains the same 
because the nonlinearity of the medium creates higher harmonics and a steeper 
pulse, whereas dispersion tends to broaden the pulse. A balance between the 
two is reached, and a stable pulse results. Thus, a communication system 
using solitons might be advantageous. Electrical distributed lines have an 
obvious direct application to integrated circuit parametric amplifiers, har- 
monic generators, and shock-wave generators for pulse shaping. They also 
have applications to secret, or secure, coding systems using two soliton 
interactions, and to data transmission using solitons. 


I. INTRODUCTION 


The study of nonlinear wave propagation along distributed electrical 
and mechanical transmission lines is important because these lines 
serve as useful models for nonlinear wave motion with dispersion in 
many interesting physical systems. Recently, nonlinear wave motion 
with dispersion has been shown to produce solitons in optical fibers.' 
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To design devices with exotic properties, one often needs exotic 
materials. It is very costly in time and money to try to produce these 
by strictly trial and error methods. Thus the usefulness of models 
becomes clear. In addition, they permit us to check quickly the 
accuracy of our computer solutions to certain partial differential 
equations that describe propagation in nonlinear materials. 

Of course, the electrical distributed lines have an obvious direct 
application to integrated circuit parametric amplifiers, harmonic gen- 
erators, and shock-wave generators for pulse shaping.” They also have 
applications to secret, or secure, coding systems using two soliton 
interactions, and to data transmission using solitons.*** In this paper 
we study soliton propagation in materials by means of electrical and 
mechanical models, and numerical solutions to the appropriate, non- 
linear, partial differential equations. Particular attention is paid to 
multisoliton interactions and the Fermi, Pasta, and Ulam problem. 

To a large extent, the material considered in this paper is that which 
seems appropriate to the materials scientist, and we extract liberally 
from the literature. This paper is largely concerned with elementary 
videosolitons, while a proposed paper, with modulated waves in non- 
linear dispersive media, is related to the interesting works of Hasegawa 
and Tappert,°’ and to more advanced videosoliton topics. 


1.1 The Fermi, Pasta, and Ulam study of nonlinear problems 


In 1955 Fermi and co-workers Pasta and Ulam did a remarkable 
analysis of nonlinear problems. They studied a one-dimensional dy- 
namical system of 64 particles with nonlinear forces between neigh- 
bors. In particular, they examined quadratric, cubic, and broken linear 
types of forces. The behavior of the system was studied for times long 
compared to the oscillation periods of the related linear problem. Their 
primary aim was to establish, by means of a computer analysis, the 
rate of approach to equipartition of energy among the various modes. 
The nonlinear terms they chose were quite small compared to the 
linear ones, usually about 10 percent smaller. Basically, they had a 
string with fixed ends, whose restoring force contained higher-order 
terms. 

For the familiar linear problem, if the initial position of the string 
is sinusoidal, it will oscillate in this mode forever. Their interest was 
to watch the string get into complicated shapes because of the nonlin- 
ear forces, and eventually, after a sufficient time, get into shapes where 
all the Fourier modes would be equally important. We now quote from 
their paper: 

Let us say here that the results of our computations show features which were, 

from the beginning, surprising to us. Instead of a gradual, continuous flow of energy 


from the first mode to the higher modes, all of the problems show an entirely 
different behavior. Starting in one problem with a quadratic force and a pure sine 
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wave as the initial position of the string, we indeed observe initially a gradual 
increase of energy in the higher modes as predicted (e.g., by Rayleigh in an 
infinitesimal analysis). Mode 2 starts increasing first, followed by mode 3, and so 
on. Later on, however, this gradual sharing of energy among successive modes 
ceases. Instead, it is one or the other mode that predominates. For example, mode 
2 decides, as it were, to increase rather rapidly at the cost of all other modes and 
becomes predominant. At one time, it has more energy than all the others put 
together! Then mode 3 undertakes this role. It is only the first few modes which 
exchange energy among themselves and they do this in a rather regular fashion. 
Finally, at a later time mode 1 comes back to within one percent of its initial value 
so that the system seems to be almost periodic. All our problems have at least this 
one feature in common. Instead of gradual increase of all the higher modes, the 
energy is exchanged, essentially, among only a certain few. It is, therefore, very 
hard to observe the rate of ‘thermalization’ or mixing in our problem, and this was 
the initial purpose of the calculation.® 


Figure la is taken from this remarkable paper. The horizontal axis 
is time, the vertical axes is energy, and the number specifies a partic- 
ular mode. Certainly there is little, if any, tendency towards equipar- 
tition of energy among the modes at a given time, and thus there is 
no mixing. 

Figure 1b, which is also taken from the same paper, shows the actual 
shapes of the string at various cycles of oscillation. (The number is 
the total number of oscillation cycles.) The initia] displacement, as 
before, is sinusoidal. What is interesting here is that after many, many 
cycles the sinusoidal shape again appears. 

Again we quote: 


It is not easy to summarize the results of the various special cases. One feature 
which they have in common is familiar from certain problems in mechanics of 
systems with a few degrees of freedom. In the compound pendulum problem one 
has a transformation of energy from one degree of freedom to another and back 
again, and not a continually increasing sharing of energy between the two. What is 
perhaps surprising in our problem is that this kind of behavior still appears in 
systems with, say, 16 or more degrees of freedom.® 


Actually, the motion is stranger than that just described. Further 
work by Tuck and Menzel? has shown that the return of the string to 
its original state is not complete on the first return cycle, because 1 or 
2 percent of the energy remains in higher modes. After eight return 
cycles the deviation is 8 percent, or worse than a single return cycle. 
Strangely enough, after 16 return cycles it is nearly in the original 
state! 


1.2 The observation of Scott-Russell 


J. Scott-Russell in 1844 published a paper entitled “Report on 
Waves” in the Proceedings of the Royal Society of Edinburgh. We 
quote from his paper: 


I was observing the motion of a boat which was rapidly drawn along a narrow 
channel by a pair of horses, when the boat suddenly stopped—not so the mass of 
water in the channel which it had put in motion; it accumulated round the prow of 
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Fig. 1—(a) Energy in each of the first five modes. Initial form of string was single 
sine wave. Higher modes never exceed 20 units in energy. (b) Actual shape of string at 
various times. 


the vessel in a state of violent agitation, then suddenly leaving it behind, rolled 
forward with great velocity, assuming the form of a large solitary elevation, a 
rounded, smooth and well-defined heap of water, which continued its course along 
the channel apparently without change of form or diminution of speed. I followed 
it on horseback, and overtook it still rolling on at a rate of some eight or nine miles 
an hour, preserving its original figure some thirty feet long and a foot to a foot and 
a half in height. Its height gradually diminished, and after a chase of one or two 
miles I lost it in the windings of the channel. Such, in the month of August 1834, 
was my first chance interview with that singular and beautiful phenomenon... . 


Thus, we have another example of the remarkable properties of 
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nonlinear wave motion. In 1895 Korteweg and deVries"! developed 
their famous equation for shallow-water waves. This equation includes 
both nonlinear and dispersive effects. One form of it is’® 


U; — 6UU,. + U2, = 0; (1) 


This very important wave equation, commonly called the K-dV equa- 
tion, appears in many branches of physics, including plasma physics, 
fluid dynamics, and solid-state physics. Solutions to the K-dV equation 
are the solitary waves described by Scott-Russell. 

The variable U represents the perturbation of a physical quantity 
from its equilibrium value. Its change in time, U,, is influenced by 
dispersion, U,,;, which distorts and spreads it, and by nonlinearity, 
UU,, which sharpens and steepens it. If these terms balance, we have 
a soliton solution. This is clearly a special solution to the partial 
differential equation. Surprisingly, this solution, or soliton, is fairly 
easy to launch. 


Il. SOLITONS 


We can introduce the concept of a soliton by considering the giant 
solitary wave described so eloquently by Scott-Russell. We look for 
traveling-wave solutions to the Korteweg and deVries equation of the 
form!” 


U(x, t) = U(x — ct) = U(E). (2) 
This corresponds to going to a steady moving reference frame with 
velocity c. 
It is clear that 
0 od 
ax dé (3) 
and 
0 d 
at = —-c dé (4) 
Thus we find that 
Vex: _ (6U _ c)U; = 0. (5) 


We see that the partial differential equation has been reduced to an 
ordinary differential equation. The soliton waves are given by 


U(x, t) = —Ya*sech?['4a(x — x, — a°t)], (6) 


which can be checked by substitution. In this expression, x, is the 
location of the symmetrical wave at t = 0. We note that this wave 
moves to the right with a velocity a®. We also note that the amplitude 
of the wave is proportional to a? and that the width is proportional to 
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a. Since the speed of the wave depends upon its amplitude, a taller 
wave can overtake a shorter one moving in the same direction. 

One might expect that when the two waves interact, because of the 
nonlinearity, there will be a massive altering of their shapes and 
speeds after the collison. This is not the case! The surprising result is 
that they emerge from the interaction completely preserved in shape 
and speed, with only a shift in position relative to where they would 
have been had no interaction taken place. Thus, in ferreting out 
solitons, we first determine whether a solitary wave exists, then we 
see if these waves retain their shape and velocity after a collision. 
Scott!® has given some working definitions for solitary waves and 
solitons. 


Definition: A solitary wave, ¢,:(&), is a localized traveling wave. More 
precisely, a traveling wave whose transition from one constant asymp- 
totic state—as £ — — © (possibly), another as & > + o—is essentially 
localized in &. 

Definition: A soliton ¢,(x — ct) is a solitary-wave solution of a wave 
equation that asymptotically preserves its shape and velocity upon 
collision with other solitary waves. That is, given any solution ¢(x, t) 
composed only of solitary waves for large negative time, 


(x, t) ~ Y gulf) as t—>—, (7) 
J 
where & = x — ct. Such solitary waves will be called solitons if they 


emerge from the interaction with no more than a phase shift. 
We see that 


p(x, t) ~ 2 Os(E)) as t+ o, (8) 

and 
Es =x — ct + 4, (9) 

with 
6; = constant. (10) 


There are, of course, quite a few nonlinear partial differential 
equations that are interesting from the point of view of soliton theory. 
We list a few of these, along with the Lagrangian density, L, from 
which they can be derived. The Lagrangian density is useful because 
it allows us to calculate certain conserved densities. 


1. The generalized K-dV equation: 
U, + aUU, + Ug = 0 (11) 


a 


1 
2 Wan 


Wi + W.V, + ; v?, (12) 
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where 
W,=U and W,, = V. (13) 


A two-interacting-soliton solution is the following:1*"* 


_ 216 + 288 cosh(2x — 8t) + 72 cosh(4x — 64t) 


ee a[3 cosh(x — 28t) + cosh(3x — 36t)]? aa 

2. The sine Gordon equation:?° 
Un — Ug + sin U=0 (15) 
Pe : (U2) — cos U. (16) 


The quantity U often has the meaning of an angle. The following 
solution represents solitons, U,, and antisolitons, UL: 


U, = 4 tan-{exp (« =) (17) 
3. The Nonlinear Schrodinger equation:?®" 
U,, +iU,+ K| U?| U=0 (18) 
: K 
L=1/2(UU7- U*U;) + | Us|? SUT’. (19) 


4, The Born and Infeld equation:"’ 
(1 — U?) U sx + 2U,0:0% — (1 + UU = 0 (20) 
L = (1 + U2 — U?)*. (21) 


We should point out that the simplest partial differential equation 
that has soliton solutions is 


Ue zi 1/c?U = 0. (22) 


It is dispersionless and linear in contrast to those described earlier, 
which contained dispersion and nonlinear terms. It is important to 
realize that if the propagating medium is linear and dispersive, or 
nonlinear and dispersionless, solitons cannot exist. 


2.1 The Toda lattice'®-?? 


Weare, of course, all familiar with a one-dimensional lattice of mass 
points. For a vibronic analysis these points are imagined to be con- 
nected by springs. In the case of large vibrations they can show 
nonlinearities. Toda considered a one-dimensional lattice with a po- 
tential of the form 
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; [exp(y; — yj-1)] + aly; — ¥-1) — - (23) 


Subsequently, Flaschka”*”4 showed that this lattice was a finite- 
dimensional analogue of the K-dV equation. Also, certain integrals of 
the Toda equations are the counterparts of the conserved quantities 
of the K-dV equation. 

The Toda-lattice equations can be written as 


d’y, 
m 2 = alexp(—br,) — exp(—brn+1)], (24) 
where 
ln = Yn — Yn-1- (25) 
The Lagrangian is 
L=y Bo — - exp(—br,) + ; exp(—brns). (26) 


We should remark that a series of mass points coupled by springs has 
the electrical analogy of a series of inductor-capacitor, low-pass cir- 
cuits. 

Toda obtained analytical solutions to the equations of motion and 
discovered lattice solitons, i.e., solitons in a nonlinear lattice. He 
proved that these solitons passed through one another without losing 
their identity. The solitary-wave solution he found was 


1 m 


Saas 


p’sech? E (Kn — Bt) + i (27) 


where ¢ is the force between mass points, K and 6 are constants, and 
6? = (4ab/m)sin h?(K/2). (28) 


2.2 Solitons in the Morse and other lattices?>-7* 


The Morse potential is of the form 


[exp(y; — yj-1) — 1]. (29) 


Rolfe et al.”° have studied solitons in this lattice by numerical methods. 
The shape of the solitons is nearly Gaussian. An initial pulse very 
quickly separates into pairs of pulses that travel in opposite directions 
and appear to propagate indefinitely. That they are indeed solitons 
can be verified by studying the collisions between the pulses. As 
required, there is no change except for a phase shift. In general, the 
Morse-soliton shape is well approximated by the Toda-soliton shape. 

Another family of lattice potentials is the power-law family, which 
can be specified as M-N, where M is the attractive power and N the 
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repulsive power. For example, 6-12 is the well-known Lennard-Jones 
potential, which is 6-power attractive and 12-power repulsive. Another 
one is the 6-32, or “screw”, lattice. 

It is very striking that the soliton shapes and behavior are all very 
nearly the same. Thus the Toda soliton is ubiquitous! This may be 
because of the similarity of the repulsive walls. 


Ill. LINEAR TRANSMISSION LINES 


Consider the transmission line shown in Fig. 2. We have all endlessly 
analyzed this circuit or variations of it in elementary physics, mathe- 
matics, or electronics courses. We recognize immediately that the 
inductance, L, and the capacitance, C, are analogous to mass, M, and 
force constant, K, of a lattice. Sometimes we look upon such a line as 
having an inductance-per-unit length, L, and a capacitance-per-unit 
length, C. Then, instead of a sequence of low-pass filters, we have 
something akin to a coaxial cable. Likewise, in the mechanical case, 
instead of a one-dimensional lattice we have a string. We might also 
say that we have a string with point masses on it, which in the limit 
becomes a uniform string. It is important to realize that a uniform 
vibrating string supports an infinity of modes, while a string with 
point masses on it “gets into trouble” if the frequency is too high. The 
same is true in the electrical case. 

The partial differential equation of a uniform”? distributed line is 


—V, = LI, (30) 
—I, = CV, (31) 
or 
Va=LCVis (32) 
with solutions of the form”? 
V = f(VLC x + t). (33) 


L = INDUCTANCE 


C = CAPACITANCE 


Fig. 2—Linear transmission line consisting of a sequence of inductors, L, and capac- 
itors, C 
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This is so familiar that it need not be pursued further. If, however, 
the capacitance or inductance is nonlinear, then new phenomena arise, 
for example, shock waves. 


3.1 Shock waves in nonlinear transmission lines 


Shock waves are familiar to all of us. We need only think about the 
sonic boom of the Concord supersonic jet or the firing of a rifle. In 
fact, even the noise caused by an auto collision on a high-speed 
highway is a shock wave. 

Consider now the nonlinear transmission line shown in Fig. 3. Here 
the capacitance of c is a function of voltage. Let us again consider the 
capacitance and inductance to be distributed. Then the partial differ- 
ential equation becomes 


-V, = -LI; (34) 
—I, = —C(v) V;. (35) 


In analogy, with the solution of the linear line, we guess that the 
following is the solution: 


V = f(VLC(v) x + t). (36) 


This can be shown to be correct.*° Thus, we see that the velocity of a 
disturbance is a function of its amplitude. In fact, we expect that the 
higher-voltage parts will travel at a different speed than the lower- 
voltage parts. 

One form of a nonlinear capacitance is the reversed bias p-n diode. 
In this case the larger the voltage, the lower the capacitance. This 
implies that the higher voltage peak of a waveform will travel faster 
than the lower voltage bottom. Thus, given enough distance or time, 
the peak can overtake the bottom and a voltage shock can develop. 

Shock-wave formation is one of the more interesting phenomena 
found in the study of wave propagation through nonlinear media. 
There are quite a few papers in the literature describing shocks on 

C(v) = VOLTAGE-DEPENDENT CAPACITANCE 


Fig. 3—Nonlinear transmission line, with capacitor, C, a function of voltage. 


L= INDUCTANCE 
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nonlinear, and nonlinear and dispersive, transmission lines.* In fact, 
there are even books devoted entirely to this topic. One way of looking 
at shock waves is to imagine that the nonlinearities are constantly 
building up higher and higher harmonics of the propagating wave. 

One of the problems involved in shock-wave studies on transmission 
lines, and in other nonlinear systems as well, is impedance matching. 
In general, a line with only one nonlinearity, inductive or capacitive, 
generates a reflected wave, which reduces the shock amplitude. How- 
ever, if both the inductance and capacitance are nonlinear functions, 
and, in particular, if L(é) = constant x C(é), then the characteristic 
impedance is a constant. 

Fallside et al. have described a line with*)*? 








Gs 
C= TV VI] (37) 
and 
L, 
L= TT (38) 


In a region of forward simple waves this line has a constant impedance 
Zo, given by 


Zo = (L./C.)” (39) 
and phase velocity, U, given by 
U=v|V| Lo/Ce. (40) 


Figure 4 shows typical shock waves formed by this line. We note in 
particular the sharpness of the shock-wave front. 

Our experience with linear transmission lines suggests that the 
nonlinear partial differential equations should represent forward-trav- 
eling and backward-traveling waves. Because of the nonlinearity, the 
principle of superposition cannot be applied. A very powerful way to 
solve those partial differential equations is the method of character- 


VOLTAGE 
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Fig. 4—Typical shock waves on transmission line of Fallside et al. 
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istics.** It is widely used in the study of gas dynamics, and the 
procedure can be found in the literature. 


. 3.2 Solitons on transmission lines**-*° 


Figure 5 shows four types of wave-propagating media. In a linear- 
dispersionless media, solitary waves can exist. In a nonlinear-disper- 
sionless media, shock waves exist. In a nonlinear media with disper- 
sion, solitons and solitary waves exist. In a linear media with disper- 
sion, broadening pulses exist. Let us now focus our attention on the 
electrical analogue of the Toda lattice. 

This network consists of a sequence of LC networks with nonlinear 
capacitors; and the circuit is dispersive. The K-dV equation, as men- 
tioned earlier, is an asymptotic equation for a weakly nonlinear lattice. 
A beautiful analysis of the K-dV equation, by Zabusky and Kruskal,*° 
gives us insight into what we will see on our transmission line. The 
expected phenomena can be broken into four time intervals. 

1. Initially, the first two terms of the K-dV equation dominate, i.e.: 
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Fig. 5—Four types of wave-propagating media and the types of waves to be expected. 
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U, + UU,. (41) 


A shock starts to develop, and U steepens in regions where it has a 
negative slope. 

2. After U has become quite steep, the third term comes into play 
and prevents a discontinuity: 


Ui + Usex: (42) 


Oscillations then develop. 

3. Solitons are “shaken off”, and each propagates with a velocity 
proportional to its amplitude. 

4. After a sufficient length of time, all the solitons come together 
with a phase that can almost reconstruct the initial waveform. 

The propagation equations for the networks are:*° 


& Lg) 214i) Veal): (43) 
re) 
< Qult) = Inalt) ~ Ind), (44) 
and 
Q,(t) = C(Va(t)) Vo(t). (45) 


Here V,, is the voltage across the nth nonlinear capacitor, and J, is 
the current through the nth inductor. 


3.3 Experiments with a Toda-lattice transmission line 


Hirota® has reported his experiments with a transmission line 
having L = 22 wH, and C(v) = 27v-°* pf. Figure 6 shows that applying 
a pulse to the line results in a train of solitons. In addition, there is a 
low-amplitude oscillatory tail formed. It is clear from eq. (43) that 


+00 
| V,(t)dt = constant, independent of n. (46) 


Thus, the area under the pulse and the area under the solitons should 
be the same. This is found to be true to about 2 percent. 

Let us now consider the case of two solitons of different amplitudes 
moving in the same direction on the nonlinear transmission line. The 
larger one moves faster than the smaller one and eventually swallows 
it up. The amplitude decreases during the overlap period. Finally, the 
larger soliton emits the smaller one, which then proceeds on its way 
unaltered. 

Suppose we now consider two solitons moving towards each other 
on the transmission line. In this case both solitons have the same 
amplitude. During the overlap the amplitude increases rather than 
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VOLTAGE 


decreases as in the previous case. After the collision both solitons 
proceed on their way essentially unaltered. 

The Fermi, Pasta, and Ulam phenomena can easily be observed on 
the Toda-lattice transmission line. A sinusoidal signal is introduced 
onto the line. By observing the waveform at various points along the 
line, we can examine the influence of nonlinearity and dispersion on 
the signal. These results are shown in Fig. 7. Figure 7a shows the 
input sine wave. In Figs. 7b and c the signal is progressively decom- 
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Fig. 6—Decomposition of a pulse into a finite train of solitons. 
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Fig. 7—Fermi, Pasta, and Ulam recurrence phenomena. 
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posed into higher harmonics, or solitons. In Figs. 7d and e the decom- 
position is reversing itself. Finally, Fig. 7f is reached, where the original 
sinusoidal signal has been reconstructed. The recurrence period (or 
distance) depends very strongly on the amplitude and frequency of the 
input sine wave. The higher the frequency and amplitude, the shorter 
the recurrence distance. 


3.4 Other transmission lines 


Besides the Toda-lattice transmission line, there are a variety of 
other electrical transmission lines on which solitons can readily be 
studied. Figure 8 shows four such transmission lines. Two of these 
have nonlinear inductances, and two have nonlinear capacitances. By 
a proper choice of parameters, we can simulate the dispersion relations 
for the electron wave functions in the Kronig and Penney band theory 
of semiconductors, ion acoustic waves in plasmas, or Trivelpiece and 
Gould waves.*! Recently, Jager and Tegude” have reported a nonlinear 
transmission line that has a cutoff frequency of about 500 mHz. This 
permits very interesting studies to be performed on a short length of 
line. 

It seems quite clear that analogues can be made for other partial 
differential equations besides the K-dV and the Toda lattice. For 
example, Scott*® has given both electrical and mechanical transmission 
lines that simulate the Klein and Gordon equation. This equation is 
particularly interesting because it describes the motion of a block wall 
between ferromagnetic domains, motion of a slide dislocation in a 


L(/) = CURRENT-DEPENDENT INDUCTANCE 
C(v) = VOLTAGE-DEPENDENT CAPACITANCE 
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Fig. 8—Nonlinear transmission lines where solitons can propagate. 
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crystalline structure, and one-dimensional models for elementary par- 
ticles. 


IV. APPLICATIONS OF SOLITONS TO COMMUNICATION 
SYSTEMS**°*” 


Since a soliton will propagate along an appropriate transmission 
line with constant velocity and without change in shape, a communi- 
cation system using solitons might be advantageous. We remind the 
reader that the shape remains the same because the nonlinearity of 
the medium is creating higher harmonics and a steeper pulse, whereas 
dispersion is constraining the harmonics and tending to broaden the 
pulse. A balance between the two is reached, and stable pulse results. 

Chu and Whitbread“ have reported their experiments on a Pulse 
Code Modulation (PCM) system using solitons. A soliton is excited 
by a rectangular pulse in the transmitter. In the receiver the incoming 
pulse is sent directly to a threshold detector. No equalizer is needed, 
because a soliton suffers no distortion. 

One problem involved in a soliton transmission system is that the 
soliton sometimes has an oscillatory tail, which influences the velocity 
of the soliton following it. For an optimum system this tail should be 
suppressed. Another problem with such a system is impedance match- 
ing of the nonlinear transmission line. Chu and Whitbread solve this 
problem by terminating the line with a piecewise simulation of a 
nonlinear resistance. Figure 9 shows a block diagram of their system. 
They conclude that: 

1. It is practical to use solitons as signal carriers in a PCM com- 
munication system. 

2. The main source of jitter is an oscillatory tail, which can be 
removed. 

3. A properly designed system would incur a lower bit error rate 
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Fig. 9—Pulse code modulation transmission system employing solitons. 
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than the corresponding system using a linear, dispersive transmission 
line. 

Suzuki, Hirota, and Yoshikawa‘*** have described a multiplex sys- 
tem based upon the unique properties of solitons. If a sinusoidal signal 
is introduced into an electrical equivalent of the Toda lattice, it is 
converted into solitons. Proper choice of lattice and frequency will 
produce two solitons of different amplitudes. These individual solitons 
can then be amplitude- or phase-modulated. 

Suppose now these two modulated solitons are introduced into a 
nonlinear network. The solitons meet, overlap, and combine into a 
single carrier, which can be transmitted. This is just an example of 
the Fermi, Pasta, and Ulam phenomena. The modulation on the 
combined signal appears to be quite arbitrary, and it is very difficult 
to distinguish the two original modulating signals that comprise it. 

If the receiver is equipped with the same nonlinear network, the 
signal is converted back into the original pair of separately modulated 
soliton trains. Strange as it seems, the feasibility of this multiplex 
system has been successfully demonstrated. 

The proposed paper will discuss modulated waves in nonlinear 
dispersive media—for example, the work of Karpman and Krushkal.* 
In addition, some of the more advanced video soliton topics will be 
examined. 
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The signal levels around and within eight suburban houses were measured 
at 800 MHz. These measurements are needed in refining the requirements for 
portable-radio communication systems that can accommodate low-power ra- 
diotelephone sets. The measurements were made from an instrumentation 
van having an erectable 27-foot-high antenna. Large-scale distributions of the 
small-scale signal medians are approximately log normal. The decrease in 
median signal level with distance ranges from d~° to d~°? for the eight houses. 
Signal decreases as d~*° for the overall data set. At 1000 feet, regressions to 
signal levels range from 12.5 to 37.1 dB below free-space propagation levels 
for locations outside and locations inside on first and second floors. In 
basements, regression levels at 1000 feet range from 29 to 48.2 dB below free 
space. For the overall data set, regression signal levels at 1000 feet are 27.7 
dB below free space. For all the basements, this value is 39.6 dB. Other signal 
statistics are given in this paper. 


I. INTRODUCTION 


Portable radiotelephones that are small, sophisticated, and low 
power are feasible because of advances in solid-state integrated-circuit 
technology. The performance of systems that operate with such radio- 
telephones strongly depends on the attenuation of radio signals prop- 
agating into buildings. Portable radiotelephone systems may operate 
at frequencies near 800 MHz. 
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Suburban residential areas, characterized by discrete houses with 
densities ranging from less than one house per acre to several houses 
per acre, comprise an important environment for portable radiotele- 
phones. In the residential environment, fixed radio terminals that 
communicate with portable handsets could be placed outside houses 
at convenient locations. Such fixed terminals are referred to as port- 
able radiotelephone terminals (PORTs). 

Measurements of attenuation in and around three small buildings 
and a house were described in an earlier paper.’ This paper describes 
attenuation measured in and around eight suburban houses. The 
measurements were made from an instrumentation van parked at 
different locations ranging from 250 to 2500 feet from the houses. A 
27-foot-high erectable antenna on the van simulated an unobtrusive 
PORT antenna. Signal levels were received and recorded in the van 
from a portable signal source moved in and around the houses. Figure 
1 depicts the measurement configuration. 

Section II of this paper describes the instrumentation and measure- 
ment procedures. Section III contains the statistical results from the 
measurements. 


Il. THE EXPERIMENT 
2.1 Instrumentation 
2.1.1 Signal source 


The portable signal source is the transmit section of a modified 815- 
MHz* handie-talkie. The transmitting antenna is a half-wavelength 
coaxial sleeve dipole attached to the top of the hand-held unit. Dc 
power is provided by a self-contained nickel-cadmium battery through 
a series voltage regulator. The regulator minimizes the output power 
drift due to normal battery discharge. The transmitter output is 0.8 
watt. The output varies less than 0.3 dB and 700 Hz over continuous 
1-hour periods that include ambient temperature changes of 0°C to 
25°C. 

2.1.2 Instrumentation van 


The instrumentation van (movable PORT) is a modified motor 
home, containing a 5-kW ac generator. An uninterruptible power 
supply isolates the instrumentation from generator voltage fluctua- 
tions that otherwise could affect measurement accuracy. The van is 
shown in Fig. 2. 

Two 27-foot antenna masts are installed in pivoting mounts so they 


*The actual frequency of the measurements is 815 MHz; however, the statistical 
results are not sensitive to small changes in frequency. Therefore, when frequency is 
referred to relative to the measurements, it will be rounded to 800 MHz. 
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Fig. 1—Pictorial representation of the measurements. 


can be stored horizontally for transport and easily erected at the test 
site. They are 14.2 feet apart and are adjusted to vertical using built- 
in bubble levels. An 815-MHz collinear receiving antenna is mounted 
on one mast. A bracket on the other mast holds the 815-MHz signal 
source at the same height to provide a reference signal for calibration 
purposes. The centers of the two antennas are at the same height 
when erected. 


2.1.3 Measuring receiver 


The measuring receiver is an 815-MHz FM communications receiver 
modified to detect the received signal envelope. The receiving antenna 
is a collinear array (4 dipole elements, 5.8-dB gain over dipole, 18- 
degree vertical beamwidth) mounted vertically at the top of the tilt- 
over mast on the instrumentation van. In the receiver modification, 
as shown in Fig. 3, an 11.7-MHz Intermediate Frequency (IF) output 
is extracted before the limiter, converted down to 13 kHz, bandpass 
filtered (BWsan = 8 kHz), and linearly detected. Figure 4 shows the 
modified receiver input/output characteristics. The modified receiver 
has a —123 dBm sensitivity for 0-dB output signal-to-noise ratio (s/ 
n) from the linear envelope detector and a 45-dB measuring range 
between levels 6 dB above the noise level and 3 dB below saturation. 
This characteristic is linear within + 1 dB over 35 dB. The input/ 
output characteristics were measured before and after the field mea- 
surements to show the long-term variations in receiver performance. 
The standard deviation of the receiver noise level is also shown. The 
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Fig. 2—The instrumentation van with the receiving antenna mast and the reference 
signal mast erected. The center of the four-element collinear receiving antenna is 27 
feet above ground at the top of the mast mounted on the right side of the van near the 
front. The signal source is at the top of its reference mast mounted on the left side near 
the rear. The center of the signal-source dipole is also 27 feet above ground. 
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Fig. 3—Block diagram of the measuring receiver illustrating the modification to the 
commercial FM receiver. Also shown is the storage oscilloscope with the integral flexible- 
disk drive used for data acquisition. 


entire receiver is enclosed in a sealed brass box to provide Radio 
Frequency (RF) isolation. A variable RF attenuator (see Fig. 3) reduces 
the input signal level in 1-dB steps to prevent overloading of the 
receiver. 


2.1.4 Data acquisition 


The analog receiver output drives a 12-bit resolution digital-storage 
oscilloscope and an integral 5-1/4 inch flexible disc drive for data 
storage. The oscilloscope is set to record 2048 samples in a 20-second 
measurement period. On each disc are recorded 16 tracks of 2048 
samples each. Data for each parked position of the van, i.e., one PORT 
location, are stored on a separate disc. 

The recorded data are transferred to a desktop computer. At the 
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Fig. 4—Input-to-output characteristic of the modified receiver. The data from after 
the field measurements include the analog-to-digital conversion in the digital-storage 
oscilloscope. The data from before the field measurements were measured with an analog 
voltmeter at the output of the linear detector. 


time of transfer the following steps are performed: (1) Hand-recorded 
log information is appended to the signal-strength data. The log 
information includes such items as the address of the house, the 
position of the van, the path azimuth, the path length, and the received 
input attenuator setting. (2) The data are scaled to convert the 
recorded signal voltages to decibels relative to 0 dB at the reference 
location 14.2 feet from the receiving antenna. The scaling takes into 
account recorded dc offsets, recorded reference levels and the received 
input attenuator settings. (3) Medians and cumulative distributions 
of signal level are calculated from the scaled data. (4) The scaled data 
are stored on flexible discs within the computer for further analysis. 


2.2 Signal reference 


For calibration purposes, both pivoting masts on the instrumenta- 
tion van are used. The received signal level from the source at the 
reference distance is used as a calibration level to which all subsequent 
data measurements are referenced. The distance represents a realistic 
compromise between near-field antenna interactions and physical 
limitations on reference mast mounting. 
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Fig. 5—Relative received signal level as a function of distance, d, between the signal 
source on top of a building and the receiving antenna mounted on the 27-foot van mast 
shown in Fig. 2. The measurements were made in an open area free of obstructions. 


Relative ratio of signal level to distance from the receiving antenna 
is plotted in Fig. 5. The van with the receiving antenna erected was 
moved away from a fixed point supporting the signal source. Both 
antennas were at the same height. At the reference antenna distance, 
the received signal decreases at the 1/d? rate of free-space propagation. 
The rate of change of the signal departs from 1/d? at shorter distances 
because of near-field effects. At longer distances, ground reflection 
becomes significant. 


2.3 Procedure 


Van locations were selected using tax maps covering the immediate 
vicinity of the houses. Points were chosen approximately equally 
spaced in azimuth around each house for each of three radii at about 
400, 800, and 1600 feet. Road layout and terrain irregularities influ- 
enced the final choices of vehicle placement. 

Measurements were coordinated between the van and the measure- 
ment location over a 450-MHz voice link. The voice link comprises a 
25-watt FM transceiver in the van and a 2-watt handie-talkie carried 
by the person making the measurements. 

A typical procedure for measuring a house starts with the van parked 
at an appropriate position. The van location must be fairly level. If 
necessary, wooden ramps are put under wheels to aid in leveling. The 
portable transmitter is installed on its mast as a local reference source. 
The mast is erected and is plumbed to vertical. Similarly, the receiving 
antenna on the opposite side of the vehicle is erected and plumbed. 
Keeping both masts plumbed on the level van assures a fixed distance 
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between the reference and receiving antennas for calibration purposes. 
The linear detector Intermediate Frequency (IF) input is grounded, 
and the dc level is adjusted and recorded on a disc track. Next, the 
detector input is ungrounded and the receiver RF attenuator is ad- 
justed so the RF reference level is within the receiver operating range. 
This level, which serves as a calibration reference at the fixed 14.2- 
foot distance, is then recorded. 

The signal source is removed from the mast and taken to the house 
for signal level measurements. The unit is hand-held? at arm’s length 
for a scan height of 4.5 feet. At a selected location either outside or 
within the house, a 20-second raster scan? is made by moving the 
transmitter in a horizontal plane at 2.5 ft/s. The 4-foot-square scanned 
area consists of 12 parallel linear scans separated by 4-inch increments. 
During the scan period, 2048 data points are taken at a rate of 100 
samples per second. 

During that period, the oscilloscope is monitored to see that signal 
amplitudes are within the receiver operating range. If they are not, 
the RF input attenuation is adjusted and the scan is repeated. The 
remaining locations within the house and immediately outside are 
similarly scanned and recorded. Upon completion of the measure- 
ments, the transmitter is reinstalled on the reference mast, erected 
and plumbed, and the dc level and RF-signal reference level are again 
recorded. The closure error between beginning and ending reference- 
level recordings is usually less than 0.5 dB. If the closure error exceeds 
1 dB, the measurements are repeated. Such high closure error occa- 
sionally occurs when the transmitter battery has discharged below the 
regulation limit of the voltage regulator. 


2.4 Received signal characteristics and definitions 


Motion of the continuous wave (CW) signal source through the 4- 
foot-square areas inside and outside of houses results in small-scale 
signal variations. The variations are caused by multipath propaga- 
tion.2* Inside houses and in areas shadowed from the van, where 
propagation is dominated by reflection and scattering, the variations 
in the received signal envelope are approximately Rayleigh distrib- 
uted.’**° Received signal minima are separated by the order of one- 
half wavelength.” The medians (or means) of these small-scale varia- 
tions are approximately stationary over the small areas, but the 
medians for areas in different rooms in a house or in different houses 
can be significantly different. Thus, the signal statistics can be mod- 
eled as a combination of a small-scale quasi-stationary process (mul- 
tipath) superimposed on a large-scale process (shadowing). This model 
is like the models used for mobile radio propagation.** 

The received signal y,(t) can be represented as 
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y(t) = L(t)R(theer*®, (1) 


where w, = 27 X 815 MHz and t is time. Since a number of paths that 
are many wavelengths long are involved, as in mobile radio, it is 
reasonable to expect any value of ¢(t) to be equally likely, i.e., d(t) is 
uniformly distributed from 0 to 27. 

The envelope R(t) is Rayleigh distributed with a stationary mean 
(or median) over small-scale areas. The median of R(t) is normalized 
to unity.)”+° From Ref. 1, small areas for which R(t) departs signifi- 
cantly from Rayleigh are those with strong signal, often with no houses 
between the van and the measurement areas. These are areas for 
which L(t) is large. However, low signals are the ones that limit radio 
system performance and are thus of most concern. Therefore, depar- 
tures from Rayleigh for large signal areas will not affect the accuracy 
of the model for most system-analysis applications. 

The large-scale random variable L(t) varies from room to room, 
from house to house and from van location to van location. Reference 
1 and Section III show that the distribution of the random variable 
U(t), defined by 


U(t) = 10 logioL7(t), (2) 


can be approximated by a Gaussian (normal) distribution for most 
random collections of small areas; that is, L(t) is log-normally distrib- 
uted. Thus, the probability density of U(t) is 


foi) = = ee way, (3) 


where the standard deviation, o, is in decibels. Reference 1 and Section 
III also show that the mean, m, varies with distance, d, between the 
van and the small area as 


m « 10 logio(d”), (4) 


where n is the distance dependence exponent. 

Only a single parameter is needed to describe the small-scale Ray- 
leigh-distributed signal variation. The median can be determined if 
somewhat over half of the samples are above the measurement thresh- 
old. The mean, however, is biased by the receiver noise level unless 
significantly more than half the samples are above the threshold. 
Therefore, medians of received signal variations for the 4-foot-square 
areas characterize the small-scale signal variations at different mea- 
surement locations. The lowest median signal measured was 17 dB 
above the receiver noise level for the data presented in Section III. 
The second- and third-lowest median signals measured were 19 dB 
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and 19.5 dB above the receiver noise. All other medians were more 
than 20 dB above, with most around 40 dB above the noise. 


il. THE MEASUREMENT RESULTS 
3.1 Measurement areas 


Characteristics of the houses measured are summarized in Table I. 
A brief description of the houses follows. 

Figure 6 shows the first house, on Rambling Brook Drive. It is a 2- 
story English Tudor style with a full basement. The house is centrally 
located on a level 250- by 200-foot lot. House densities in this area are 
generally one per acre. This house is 5 years old, has aluminum siding 
on three sides, and has nonmetallic composition siding on the front. 
Aluminum combination storm window and screens are on all window 
and door openings. The metal screen covers only one-half of the area 
of each window opening. This house has full, foil-backed insulation in 
the walls. Outside this house are numerous low foundation trees. There 
is a dense row of 15-foot-high evergreens along the west property line, 
and a few large trees within an 800-foot radius. These measurements 
were made in April 1982. 

Figure 7 shows the second house, on Winding Brook Way. It is 
located on a slightly raised mound on a 1-acre lot. It is a bilevel ranch 
house. The basement is half above ground and is part of the living 
area. The house has all wood siding and nonmetallic wall insulation. 
House densities in this area are one per acre. This house is 15 years 
old. Most trees and plantings in this area are the same age. The 
foundation plantings are medium density, with the exception of several 
large evergreens on the corners and in the rear near the property line. 
Metal screening covers all the window openings halfway. These mea- 
surements were made in May 1982. 

House number 3, on Valley Point Drive, and shown in Fig. 8, is a 
very large, new, 2-story colonial with vinyl siding covering foil-faced 
sheathing. All windows are full-metal screened with the exception of 
sliding glass doors located in the rear. This house has a full concrete 
block basement and is located on a mostly level 1-acre lot. There are 
very large trees completely covering the rear property line, which drops 
off sharply into a ravine. The ravine is filled with large trees. All other 
houses in this area, although different in style, are constructed on 1- 
acre lots. For the most part, there are very few trees, except for the 
dense woods running behind the house and adjacent houses. These 
measurements were made in late May to mid-June 1982. 

Figure 9 shows the fourth house, on Courtland Lane. It is an L- 
shaped ranch house without a basement. It is on a 75- by 100-foot lot. 
All surrounding lots and houses are similar, with a housing density of 
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House Lot Size (ft) 
Rambling Brook 250 by 200 
Winding Brook 200 by 200 
Valley Point Drive 200 by 200 
Courtland Lane 75 by 100 
Alden Lane 175 by 250 
Monmouth Ave. 100 by 200 
Tallen Drive 75 by 110 
Rutledge Drive 50 by 150 


Table |—Characteristics of houses in study 


House Densities Exterior-Side 


in Area Material 
1 per acre 3 aluminum 
1 composition 
1 per acre Wood 
1 per acre Vinyl over foil 
5 per acre 3 asbestos 
1 wood 
1 per acre All brick 
0.5 to2peracre All brick 
5 per acre 3 asbestos 
1 wood/brick 


1 to 5 per acre All wood 


* No metal screens on rear sliding doors. 


Density of 
Foundation 
Plants 


Avg. 
Avg. 


Avg. 
Avg. to high 


Avg. to low 
Avg. 
Avg. 


Avg. 


Large Shrubs or 
Trees in Area 


Few 
Avg. 


Many in rear 
Avg. 

Few, some large 
Few, very large 
Few 


Few 


Metal- 
Window 


Screening Wall-Insulation Backing 


Half 


Half 
Full* 
Half 


Half 
None 
Full 


Half 


Foil 
Paper 


Foil 

Paper 

Paper 

Metal lath under plaster 
Paper 


Unknown 
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Fig. 7—The house on Winding Brook Way. 
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Fig. 8—The house on Valley Point Drive. 
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Fig. 10—The house on Alden Lane. 


about five per acre. This house has wood siding on the front and 
composition siding on three sides. Half of each window opening is 
covered with a metal screen. Wall insulation is nonfoil glass wool. 
There are some fairly large shrubs along the foundation and some 
medium-sized trees on the lot. These measurements were made in 
August 1982. 

Figure 10 shows house number 5, on Alden Lane. It is an all-brick 
construction with a full basement. It is centrally located on a level lot. 
This house does not have foil-backed insulation and has metal half- 
screened windows throughout the house. It has two large, unscreened 
windows, one front and one rear. House densities in this area are 
generally one per acre. There are medium-height foundation plantings 
and some medium to large trees near the house. There is a large open 
area behind a 100-foot strip of heavy woods to the back. This strip is 
perpendicular to the lot. These measurements were made in late 
August to early September 1982. 

Figure 11 shows house number 6, on Monmouth Ave. It is a large, 
all-brick, 2-story colonial with a full basement and is on a half-acre 
lot. This is an older house with metal-lath plaster walls. Vegetation 
density is medium around the house. The property has numerous large 
trees in the rear and one or two large trees in the front. Most streets 
in the area are tree-lined with older shade trees. This house has full- 
metal screening on its average-sized windows, but two very large front 
first-floor windows are unscreened. House densities in this area vary 
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Fig. 12—The house on Tallen Drive. 


considerably, but generally run from one-half to two per acre. These 
measurements were made in October 1982. 

Figure 12 shows house 7, on Tallen Drive. It is located on a 75- by 
110-foot lot in an area of homes that are all on quarter-acre lots. 
Siding on this house is wood and masonry on the front and asbestos 
on three sides. This house has full-metal screening on the front 
windows and partial screening on the few side windows. Wall insula- 
tion has no foil backing. Most of the foundation plantings are average- 
sized, and there are a few trees across the rear property line. The 
surrounding properties generally have a few medium-sized trees, which 
had just dropped their leaves. These measurements were made in mid- 
November 1982. 

The last house measured, on Rutledge Drive, is shown in Fig. 13. 
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Fig. 13—The house on Rutledge Drive. 


This is a very old house located on a 50- by 150-foot lot. It is a wood- 
sided 2-story colonial with a partial basement. Wall insulation material 
is unknown, but there are suggestions that the newer two-thirds of 
this house may have foil-backed insulation. Window sizes in this house 
are average and all are covered halfway with metal screening. House 
densities in this area are generally two or three per acre. This house 
has medium-to-large foundation plantings. It also has a few large trees 
on the property and in the area. There are two 30-foot evergreens, 
very dense, directly in front and a large wooden 2-car garage directly 
across the back of the house. These measurements were made in mid- 
December 1982 to January 1983. 


3.2 Large-scale statistics of small-scale medians for individual houses 


The data points in Figs. 14 through 21 are the medians of the 
measured signal envelopes from small-scale areas for each of the eight 
houses. That is, the data points are samples of the random variable 
L(t) in eq. (1). Each data point represents the median level of 2048 
samples of R(t) taken within a 4-foot-square area at a particular 
location. The signal levels are in decibels relative to 0 dB at the van 
reference. The medians are plotted versus the distance between the 
van antenna and the area. Note that the independent variable, dis- 
tance, is on the ordinate. The dotted line on each figure represents 
free-space (FS) propagation (d~*) relative to 0 dB at the van reference. 
The measurement locations are outside (OS) the houses, in rooms on 
the first (1) and second (2) floors, and in the basements (B), as 
indicated by the different symbols on the figures. The outside locations 
are about 5 to 10 feet in front of the midpoints of the four outside 
walls of the houses. 
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A strong dependence of signal level on distance is evident in the 
data in all the figures. The straight lines, except for the free-space 
line, are linear-least-squares regression lines through the different 
groups of data as indicated. The regression lines are determined using 
distance as the independent variable. These lines are a good quanti- 
tative measure of the strong distance-dependent trends. The regression 
slopes yield the distance-dependence exponents, n, in eq. (4). One 
average of the median signal level for each grouping of data is given 
by the level at 1000 feet taken from each regression line. The distance- 
dependence exponents and the levels at 1000 feet are indicated on 
each figure. The exponents and the 1000-foot levels with the —37 dB 
free-space level removed are summarized in Table II. 


Table II—Parameters for individual houses (vertically polarized, 
27-foot antenna height) 


1000 ft to 1000-ft 
Relative Building 


Distance Free Space Attenua- No. of 
Floor Exponent (dB) tion (dB) a (dB) F Points 
Rambling Brook Drive 
OS* —4.6 —12.5 — 8.8 51 35 
1 —4.0 —18.6 6.1 7.3 54 36 
2 —3.0 —16.6 4.1 4.8 54 raf 
Bt —3.2 —29.0 16.5 5.4 13 9 
Winding Brook Way 
OS —3.4 —20.7 _— 6.2 55 40 
1 —3.7 —24.0 3.3 5.4 42 20 
2 —3.3 —21.4 0.7 6.0 55 40 
Valley Point Drive 
OS —5.3 —16.4 — 9.9 37 20 
1 —6.2 —28.5 12.1 7.3 65 15 
2 —5.7 —26.6 10.2 6.4 104 21 
B —5.5 —37.7 21.3 6.3 16 5 
Courtland Lane 
OS —6.1 —31.6 _— 5.5 220 36 
1 —6.0 —-34.6 3.0 5.3 341 54 
Alden Lane 
OS —5.8 —32.7 — 6.0 260 40 
1 —5.8 —37.1 4.4 6.6 251 50 
—6.2 —48.2 15.5 5.0 89 10 
Monmouth Avenue 
OSs —4.8 —27.6 — 5.5 115 36 
1 —4.6 —32.9 5.3 6.7 34 18 
2 —§.2 —35.4 7.8 6.3 77 27 
B —4,2 —42.6 15.0 6.0 16 9 
Tallen Drive 
OS —4.7 —27.7 —_ 5.1 165 40 
1 —4.7 —30.5 2.8 4.6 202 40 
B —4,7 —39.7 12.0 4.1 121 20 
Rutledge Drive 
OS —4.1 —23.5 — 5.1 91 35 
1 —4.2 —29.1 5.6 4.3 107 27 
2 —3.8 —24.6 1.1 5.7 32 18 
B —4,3 —38.7 15.2 3.6 44 9 


*OS = Outside. 
TB = Basement. 
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An average attenuation at 1000 feet attributable to a building can 
be defined as the difference between the regression-line value outside 
and the regression-line value for the particular grouping of the data 
inside. For example, from Fig. 14 and Table II, the signal level outside 
at 1000 feet is —49.5 dB and the level inside for the first floor is —55.6 
dB. Thus, the building attenuation for the first floor for the house on 
Rambling Brook Drive is 6.1 dB. Building attenuation values are also 
summarized in Table II. 

After the regression-level trend values are subtracted from each 
data point in Figs. 14 through 21, there is still considerable scatter in 
the resulting signal-level residuals. The standard deviations, o, of the 
residuals after removal of the trends are listed in Table II. 

The distance dependences and signal levels are different for each 
floor of the same house and are significantly different between houses. 
Because of these differences and the considerable scatter in the data, 
it is reasonable to question whether the regression lines are meaningful 
or are only a manifestation of statistical fluctuation. This question is 
addressed in the next section. 
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Fig. 14—Medians of the small-scale signal envelope variations for different measure- 
ment locations plotted versus distances between the locations and the van antenna. The 
locations are inside and outside a house on Rambling Brook Drive. 
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Fig. 15—-Signal envelope medians and regression lines for the house on Winding 
Brook Way similar to those in Fig. 14. 


3.2.1 Statistical significance 


The statistical significance of the regression lines can be assessed 
using the well-known “F” test. The F values determined during the 
regression calculation indicate the likelihood that the line could be the 
result of statistical fluctuation in a sample of the same number of 
points taken from uncorrelated data, i.e., that the result is due to 
chance. In general, the larger the F number, the more significant the 
result. Also, for larger numbers of points, the result has the same 
significance at lower F numbers. Values of F and the number of data - 
points (medians) included in the regression are also shown in Table 
II. 

All of the F values in Table II have a very high degree of significance, 
except for three of the basements. The likelihood of the result being 
due to chance is much less than 0.1 percent for all but the three 
basement cases, i.e., they are significant at 0.1 percent or better. The 
results are significant at 0.5, 1, and 5 percent for the basements on 
Monmouth Avenue, Rambling Brook Drive, and Valley Point Drive, 
respectively. The lower significance for these three basements is 
largely due to the smaller number of measurements taken in them. 
Thus, in general the regression lines are statistically significant, that 
is, if more measurements were made in the same houses from different 
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Fig. 16—Signal envelope medians and regression lines for the house on Valley Point 
Drive similar to those in Fig. 14. 


van locations or in similar houses in similar areas, the regression lines 
would remain essentially the same. The differences in the parameters 
for the different houses must then result from actual differences in 
construction and in the environments around the houses. This situa- 
tion indicates that the combined statistics from the eight houses will 
be nonhomogeneous and will depend on the number of measurements 
included from each kind of house and environment. It also indicates 
that portable-radiotelephone systems should have coverage-area sizes 
and possibly frequency-reuse factors tailored to fit the particular 
environments that are encountered. However, to make overall esti- 
mates of these system parameters, combined statistics for the eight 
houses are still useful. 


3.2.2 The effect of the ground 


Most of the distance-dependence exponents in Table IJ are less than 
—4, However, in free space the signal level would decrease as d~’. 
Some of the additional rate of decrease can be accounted for by the 
presence of the ground. 

For the distances involved here, the earth’s curvature is negligible. 
At 800 MHz, the reflection coefficient phase is approximately 180 
degrees for all polarizations for small angles between the propagation 
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Fig. 17—Signal envelope medians and regression lines for the house on Courtland 
Lane similar to those in Fig. 14. 


direction and the ground.’ For propagation between two points sepa- 
rated a distance d and at heights h, and h; above flat ground, the field 
amplitude at one point E, relative to the transmitted field amplitude 


E, is given by’ 
z ; sin | 2a (5) 


where ) is the wavelength. For large d, this approaches E,/E, a d~ or, 
for power, this becomes P,/P, a d~*. The approximation d™ is good 
for the parameters h, = 4.5 feet, h, = 27 feet, d S 1000 feet, and a 
frequency of 800 MHz. These parameters are appropriate for the 
measurements outside houses and on some of the first floors. The 
effective exponent becomes a little greater than —4 for distances less 
than 1000 feet. For second floors and for short distances, not only 
does the approximation of the sine with its argument break down, but 
also the reflection coefficient angle may depart significantly from 180 
degrees for some ground conditions. 

All of the outside and first floor exponents in Table II are less than 
—4 except for one. The house on Winding Brook Way is a little higher 
than some of the surrounding ground. Thus, even its outside measure- 
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Fig. 18—Signal envelope medians and regression lines for the house on Alden Lane 
similar to those in Fig. 14. 


ments could be more than 4.5 feet above ground. The first floor of this 
house is at ground level at the back side and underground at the front. 
For some directions, the first floor is more like a basement. Thus, the 
trends in the distance-dependence exponents for all the houses are 
consistent with the presence of the ground. The medians that are 
above the free-space level on some of the figures are also consistent 
with the presence of the ground. [Note the factor of 2, i.e., 6 dB, in eq. 
(5)]. The fact that some of the exponents are considerably less than 
—4 is probably due to the additional attenuation experienced in reflect- 
ing from and propagating through intervening houses and trees. 


3.2.3 Discussion 


From Table II and Figs. 14 through 21 it is evident that the 
attenuations into basements are significantly greater (~ 10 dB) than 
the attenuations into first and second floors. This suggests that, if 
basements are to be served by a portable radiotelephone system, the 
attenuations into basements will dominate the radio-link gain require- 
ments. 

Attenuations into the houses generally follow expectations based on 
the metallic content of the walls and other wall construction features. 
The house on Valley Point Drive, with aluminum foil in the walls and 
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Fig. 19—Signal envelope medians and regression lines for the house on Monmouth 
Avenue similar to those in Fig. 14. 


full metal screens, has the largest attenuations for all floors. The 
houses on Rambling Brook Drive, Rutledge Drive, and Monmouth 
Avenue form an attenuation grouping with about 6 dB less attenuation 
into the first floors. Two of these houses definitely have metal in their 
walls, with Rambling Brook having half metal screens, and Monmouth 
Avenue having full screens except for two very large, unscreened 
picture windows. Uncertainty in the construction of the older house 
on Rutledge Drive does not help in the attempt to explain its higher 
attenuation. The second-floor and basement attenuations generally 
follow the first-floor trends, except for the somewhat higher second- 
floor attenuation on Monmouth Avenue, and a somewhat higher 
basement attenuation on Alden Lane. All of the second-floor windows 
at the Monmouth Avenue address have full metal screens except for 
two unscreened windows in two sundeck doors. The Alden Lane 
basement appears to be lower in the ground than the other basements. 
These factors probably account for the higher attenuations. The 
attenuation for the first floor of the partially-screened brick house at 
Alden Lane is about midway between the attenuations for the metallic- 
walled houses and for the wooden- and composition-walled houses. 
The dense brick walls seem consistent with this observation. The three 
houses with wooden and composition walls on Winding Brook Way, 
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Fig. 20—Signal envelope medians and regression lines for the house on Tallen Drive 
similar to those in Fig. 14. 


Courtland Lane, and Tallen Drive form a low attenuation group with 
only about 3-dB attenuation into the first floors. The second-floor 
attenuation on Winding Brook Way and the basement attenuation at 
Tallen Drive are also lower than the similar attenuations for the other 
houses. 

The median signal levels at 1000 feet both outside and inside are 
~ not well correlated with the attenuations into the houses, i.e., with the 
differences between outside and inside levels. External environmental 
factors appear to affect the signal levels as much or more than the 
house constructions. For example, signal levels were lowest at Alden 
Lane, the brick house with only moderate attenuation. Signal levels 
were highest at Rambling Brook Drive, the second-highest attenuation 
house. The two houses in high-density housing areas, the Courtland 
Lane, and Tallen Drive locations, have low-signal levels. The houses 
with the higher-signal levels are generally in the least dense housing 
areas that are not heavily wooded. The significance of these general 
trends is difficult to assess with a small sample of eight quite different 
housing environments. It appears reasonable, however, that attenua- 
tion would be greater (signal levels lower) for propagation through 
areas with greater house densities. 
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Fig. 21—Signal envelope medians and regression lines for the house on Rutledge 
Drive similar to those in Fig. 14. 


3.3 Large-scale statistics of small-scale medians for all eight houses for 
different groupings of the data 


3.3.1 Parameters for groupings by floors 


The signal medians and distances from Figs. 14 through 21 are 
combined and separated into groups for outside, first floors, second 
floors, and basements in Figs. 22 through 25. Each symbol of these 
figures represents data from one of the houses, as indicated in Table 
III. The straight solid lines are the least-squares regression lines for 
all the data on a given figure. The longer dashed lines are regression 
lines for all the data between distances of 250 and 1250 feet. Similarly, 
the shorter dashed lines are regression lines for all the data between 
distances of 690 and 2300 feet. The dotted lines are, again, free-space 
lines. A regression for all the first- and second-floor data for all 
distances, i.e., from 250 to 2300 feet, is plotted as a long and short 
dashed line and a dashed and dotted line on Figs. 23 and 24 for 
comparison with the line for the individual floors. The nonhomogene- 
ity of the data discussed in Section III is evident in Figs. 22 through 
25. 

The distance-dependence exponents, the signal levels at 1000 feet 
and the building attenuation at 1000 feet are listed in Table IV for the 
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Table !!1—Symbol legend for Figs. 22 
through 25 
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Fig. 22—Medians of the small-scale signal envelope variations for different measure- 
ment locations plotted versus distances between the locations and the van antenna. The 
locations are outside of the eight houses. Signal levels are with respect to 0 dB at the 
signal reference. Regression lines are drawn for all the outside data from all eight houses 
for all distances and for two subsets of the outside data. 


regression lines in Figs. 22 through 25. The small negative values for 
building attenuation at 1000 feet for the second floors merely indicate 
that the average of the median signal levels for the second floors were 
slightly larger than the average of the levels outside at 4.5 feet above 
ground. This is reasonable since the second floors are less obstructed 
by intervening houses and are affected somewhat differently by the 
ground [see eq.(5)]. The # parameter and number of data points in 
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Fig. 23—Signal envelope medians and regression lines for the data from the first 
floors for all eight houses similar to those in Fig. 22. In addition, the regression line is 
included for all the first- and second-floor data combined. 


each regression are also listed in the table. For these data groupings 
by floors, the regression is significant well below the 0.1-percent level 
readily available in F tables.® Thus, even though the data sets are 
nonhomogeneous, the regressions to the data appear very good. Table 
IV also includes the standard deviations, o, of the residual signal levels 
after subtracting out the regression-line values at the same distances, 
i.e., after removing the trends. 

All the signal medians for all houses and all floors (including outside 
data) are plotted in Fig. 26. Because of the high density of points, no 
attempt has been made to separate any of the data groupings in that 
figure. The solid regression line is for all the data. The dashed line is 
for all of the floors for distances between 250 and 1250 feet. The 
regression line for all floors and for distances between 690 and 2300 
feet is indistinguishable from the solid line for all the data. Parameters 
for all the floors combined are also listed in Table IV. The F values 
for these data groupings are also significant well below 0.1 percent. 

The distance-dependence exponents are generally less than —4 and 
range down to —5.5. The overall value for all the data is —4.5. In 
general, for all the groupings by floors, the signal levels decrease faster 
with distance for the first few hundred feet than for the greater 
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Fig. 24—-Signal envelope medians and regression lines for the data from the second 
floors for the five houses that have second floors. The figure is similar to Fig. 22. In 
addition, the regression line is included from Fig. 23 for all the first- and second-floor 
data combined. 


distances. This is reasonable since there are few, if any, intervening 
houses and trees in the first few hundred feet from a house. After 
several hundred feet there are many intervening houses and trees. 

It is evident from Table IV and also generally from Table II that 
the signal levels and building attenuations for the various floors are 
different. As we discussed earlier, the second floors generally experi- 
ence higher signal levels and lower attenuations than the first floors. 
The basements experience much lower signal levels and much higher 
attenuations than first floors. In fact, the basement attenuation is so 
much greater that basements will probably need to be treated sepa- 
rately in considering portable-radio-system performance. 

The standard deviations of the residual signal levels are generally 
between 7.5 dB and 10.5 dB for the groupings in Table IV. The purpose 
of removing the trends for the data groupings is to reduce the standard 
deviations of the scatter in the signal levels, and thus, to permit tighter 
estimates of system performance. In general, removal of the regression 
line trends from these groupings significantly reduces the standard 
deviations. For example, before the trend is removed from the group 
containing all the data, the standard deviation was 14.9 dB. After 
removal of the trend it is the 10 dB listed in Table IV. This significant 


RADIO-WAVE ATTENUATION 947 


3000 


2000 








kK 
Ww 
Ww 
uw 
z 1000 
Ww 
9 i 
< Oo 4 
D + J = weeeee FREE SPACE 
(a) 
re REGRESSION LINES 
a/b (ALL EIGHT HOUSES) 
500 
5 BA ALL DISTANCES 
OF a ——— 250 TO 1250 FEET 
—~—-— 690 TO 2300 FEET 
VERTICALLY POLARIZED, 
: ‘ 27-FOOT ANTENNA HEIGHT 
300 : FLOOR B 


-20 -30 -40 -50 -60 -70 -80 -90 -100 -110 
SIGNAL LEVEL RELATIVE TO REFERENCE IN DECIBELS 


Fig. 25—Signal envelope medians and regression lines for the data from the base- 
ments for the six houses that have basements. The figure is similar to Fig. 22. 


reduction in standard deviation is essentially the fact indicated by 
high significance of the F parameter. 


3.3.2 Cumulative distributions for different floors and distance groupings 


The solid lines in Figs. 27 through 30 are the cumulative distribu- 
tions of the residuals for all houses and all distances for the floor 
groupings indicated. The residuals are the levels remaining when the 
regression-line trends are removed from the signal medians, as we 
discussed earlier. 

Also plotted in Figs. 27 through 30 are distributions for two other 
distance groupings of the data. These other two groupings contain the 
residuals for the shortest distances, i.e., between 250 feet and 690 feet, 
and for the longest distances, i.e., between 1250 feet and 2300 feet. 
For these distance groupings, and for the intermediate distances of 
690 feet to 1250 feet, the distance-dependence trends removed were 
those trends determined for all distances for the appropriate floor 
groupings. That is, the trends removed were the solid lines on Figs. 
22, 25, and 26 for outside, for basements and for all data and the 
dashed and dotted lines on Figs. 23 and 24 for first and second floors 
combined. The straight lines drawn on Figs. 27 through 30 represent 
log-normal distributions having the standard deviations of the various 
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Table IV—Parameters for all eight houses (vertically polarized, 27-ft 
antenna height) 
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Fig. 26—Signal envelope medians and regression lines for all the data from all the 
houses for all the floors and outside combined. All the medians are plotted as “o” with 
no distinction as to house. 
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Fig. 27—Cumulative distributions of the medians of the small-scale signal envelope 
variations for outside locations after the trend values for the solid regression line in Fig. 
22 are subtracted out. 


data groupings. Standard deviations for the three distance groupings 
with the overall trends removed are listed in Table V. 

The mean value will be zero for the residuals of a data set after 
removal of the regression trend for that particular data set. The mean 
of the residuals of a subset of a data set after removal of the trend of 
the entire set will not necessarily be zero. The closeness of the mean 
of the subset residuals to zero is an indication of how well the trend 
of the entire set represents the trend remaining in the subset. The 
means of the various distance groupings (subsets) after removal of the 
trend for all distances (sets) are also listed in Table V. Most of the 
means are less than 1 dB, with the largest being 1.5 dB. Considering 
the much larger standard deviations, these small means indicate that 
the overall trends are quite applicable to the distance groupings also. 

The cumulative distributions of the measured data in Figs. 27 
through 30 are all within +2 dB of the associated log-normal distri- 
bution with the same standard deviation for the range from 10 percent 
to 90 percent. Most of the measured distributions are within +1 dB 
over the 10-percent to 90-percent range. The fit of the measured 
distributions is worse in the tails, where statistical fluctuations are 
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Fig. 283—Cumulative distributions for first- and second-floor locations combined 
similar to those in Fig. 27. The regression trend removed was the dashed and dotted 
line in Figs. 23 and 24. 


expected to be worse, because of the small number of samples. Thus, 
log-normal distributions are good descriptions for the various group- 
ings of the data. 

For the different distance groupings in Table V, the standard 
deviations for all the groupings by floors increase with increasing 
distance. The increase is small but is very consistent across all the 
different groupings. Therefore, increase in standard deviation with 
distance probably represents an actual characteristic of the propaga- 
tion environment. 


IV. SUMMARY 


Signal levels were measured within and around eight suburban 
houses from various locations of an instrumentation van. The van 
locations ranged from 250 to 2500 feet from the houses. Median signal 
levels were determined for the small-scale signal variations in 4-foot- 
square areas. Parameters describing the variation of these small-scale 
medians over large-scale changes for the eight individual houses are 
summarized in Table II. Parameters describing large-scale changes for 
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Fig. 29—Cumulative distributions for basement locations similar to those in Fig. 27. 
The regression trend removed was the solid line in Fig. 25. 


Table V—Parameters for all eight houses (vertically polarized, 
27-foot antenna height) 


o Trend Re- Mean Trend 
Floor moved (dB) Removed (dB) No. of Points 
250 to 690 ft 
OS 71.8 0.4 67 
1 6.1 0.6 61 
2 7.2 —0.7 31 
B 3.9 0.2 16 
1 and 2 6.4 0.4 92 
OS, 1, 2,B 8.0 0.3 175 
690 to 1250 ft 
Os 9.0 —1.5 91 
1 8.5 —1.5 92 
2 8.0 —0.2 35 
B TA —0.5 17 
1 and 2 8.8 —1.5 127 
OS, 1, 2, B 9.5 -1.3 235 
1250 to 2300 ft 
Os 10.3 0.9 124 
1 9.8 0.9 107 
2 10.3 0.4 66 
B 9.0 0.2 29 
l and 2 10.7 0.9 173 
OS, 1, 2,B 11.2 0.8 326 
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Fig. 30—Cumulative distributions for all floors and outside combined similar to those 
in Fig. 27. The regression trend removed was the solid line in Fig. 26. 


different groupings of the data from all eight houses are summarized 
in Tables IV and V. Effects of house construction and the environment 
are discussed in Section 3.2.2. 
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Transmission Errors in Companded PCM Over 
Gaussian and Rayleigh Fading Channels 
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Transmission errors in y-law pulse-code-modulated systems are capable of 
inflicting considerable distortion in the recovered analog signals. We analyze 
this distortion when the transmission is over radio channels, and when either 
noncoherent frequency shift keying or coherent phase shift keying (CPSK) 
modulation is employed. We consider both Gaussian and Rayleigh fading 
channels, and for the latter we investigate the effect of bit scrambling prior to 
transmission and two types of diversity. We determined the gain in overall 
signal-to-noise ratio due to using the minimum distance code compared to the 
conventional folded binary code for low values of input signal power. In 
addition to the theoretical results, we present computer simulations of y-law 
pulse-code-modulated encoded speech transmitted over the same channels, 
using CPSK modulation, bit scrambling, and diversity where appropriate, i.e., 
for the same conditions as employed in the derivation of our theory. 


I. INTRODUCTION 


Pulse Code Modulation (PCM) is well entrenched in the telephone 
networks as a means of conveying speech in a digital format,’ and its 
impairments are therefore worthy of serious investigation. The distor- 
tion in the recovered speech signal arises from noise in the encoding 
process, and the presence of digital transmission errors. By employing 
logarithmic companded PCM at bit rates of 56 kb/s or 64 kb/s, and 
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by operating at a Bit Error Rate (BER) less than 107°, we obtain so- 
called toll-quality speech.” For channels where the BER significantly 
exceeds 10°, for example, radio channels, the distortion in the re- 
covered speech due to transmission errors becomes significant, domi- 
nating the encoder noise. The distortion due to transmission errors is 
known as digital noise,** and can be combatted by means of channel 
coding,”® weighting the amplitude of the binary bits prior to transmis- 
sion,’” or by post-enhancement procedures.!?"” 

In this study we are concerned with determining the digital noise 
power in yp-law PCM radio transmission systems. We examine this 
power for different radio channels, methods of Radio Frequency (RF) 
modulation, types of binary encoding, bit scrambling, and the appli- 
cation of diversity. In Section II we quantify digital noise power, while 
in Section III we state our objective performance criterion. We ex- 
amine two types of channels, the Gaussian in Section IV, and the 
Rayleigh fading channel in Section V. In the latter case we investigate 
the effect of scrambling the u-law PCM bit stream prior to transmis- 
sion, and the advantages of diversity. The modulation methods con- 
sidered are Noncoherent Frequency Shift Keying (NCFSK) and Co- 
herent Phase Shift Keying (CFSK). The minimum distance code 
designed for low-level speech is described in Section VI. In Section 
VII we discuss the performance of the various p-law PCM radio 
systems, where our theoretical formulae are presented in graphical 
formats, along with simulation results for speech signals, and show 
the experimental results for p-law PCM speech transmitted over a 
mobile radio channel. In the last section we summarize our discourse 
and draw conclusions. 


Il. DIGITAL ERRORS IN PCM SYSTEMS 


Figure 1 shows a schematic form of the basic companded PCM 
system. The input sequence {x}, having a Probability Density Function 
(PDF) px(x), has its amplitude compressed according to the function 
f(x), and is then uniformly quantized. The combination of the function 
f(x) and the uniform quantizer produces an effective nonlinear quan- 
tizer, as shown in Fig. 2, where f(x) and the number of quantization 
levels have been selected arbitrarily for the purpose of illustration. 
The voltage ranges of the input and quantized signals are normalized 
for convenience to [—1, 1], and the quantized levels are uniformly 
spaced apart as a result of the uniform quantizer. However, the range 
of input amplitudes that are quantized to a particular value increases 
with the magnitude of the input levels. Thus all input amplitudes 
approximately in the ranges x; + (6;/2) and x;4; + (6;4:/2) are quantized 
to y; and yi+1, respectively, where 6;41 > 6;. 

Suppose the input signal is sampled at a rate in excess of the 
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Fig. 2—Exemplary nonlinear quantizer; arbitrary f(x) and number of quantization 
levels. 


Nyquist rate, and the resulting sequence {x} is quantized to {y}. Binary 
encoding of the quantized sequence ensues to yield {L}, where each 
component is an N-bit word. The {ZL} sequence is filtered, modulates 
a carrier (e.g., using NCFSK), and is transmitted. At the receiver 
demodulation is performed, and the bits are regenerated. In Fig. 1 we 
show a particular word L; at the transmitter, and its regeneration at 
the receiver as L;,,. The word L;,; may be represented by the exclusive- 
OR (®) operation of the binary vectors L; and e;, 


Din = L; @ e, (1) 


where L;, = L; if no transmission errors occur, and e; is the [th digital 
error sequence of length N bits. Whenever the error sequence contains 
a logical 1 representing the presence of a bit error, the true bit in L; is 
inverted, and the error appears in L;,. The N-bit word L;, is binary 
decoded to y;,, and the inverse operation of f(x) (i.e., an expansion) is 
performed to yield the recovered sample, 


xin = fyi). (2) 


The recovered sample x;, differs from the input sample x, and this 
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difference is the overall system noise sample. The average noise power 
in a PCM system is, therefore, 


eA Ej{(x a, xi)"}, (3) 


where E{(-)} denotes expected value of (-) and is formulated over both 
source and channel statistics. For the large number of quantization 
levels considered here, typically 256, the average noise power may be 
separated into three components, viz: 


P= e+ et &, (4) 


where ¢2, 2, and ¢? are the quantization, clipping, and transmission 
error noise power components, respectively. The noise power generated 
in the encoder due to the amplitude and time quantization is 


eg + & = E{(x — x;:)*}, (5) 


where x; is the recovered sample associated with the quantized sample 
y; in the absence of transmission errors, and the expected value is 
formed over the source statistics. We are more concerned here with 
the digital noise power, i.e., the noise in the recovered samples due to 
the presence of digital errors, and we express this power as 


€ = ath (Xj ae xit)"}, (6) 


where E;,(-) signifies that the average of (-) is performed over all 
levels of y; and all possible error sequences e;. By observing that the 
source, @.g., a speech signal and PCM encoder, is independent of the 
imperfections in the channel, we may express «2 as 


2N-1 


e, = 2 pilii{(x; — xi1)7}. (7) 


The average E; is formed over all the possible 2% quantization levels, 
and p; is the probability of occurrence of the specific error sequence e;. 
At this juncture we introduce the A-factor,** defined as 


A, & E;{(x; — x:1)"}, (8) 


enabling us to write eq. (7) as 


2N-1 


6, = 2 pA}. (9) 
= 


The A-factor A; is the average noise power at the output of the PCM 
decoder due to the presence of an error sequence e;. We have used the 
notation €), €2, --+ , ey to represent the single error bit patterns in the 
most significant bit (msb), next msb, --- , least significant bit (Isb), 
respectively. For example, e3 is the binary sequence 00100000, implying 
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the error occurred in the third bit. However, we may have multiple bit 
errors, and | has integer values that extend to 2" — 1, since e; must 
contain at least one component that is a logical 1. The number of 
possible error patterns for w bit errors in an N-bit word is 


P - (") = MW Sen. LN ead) 


; = , (20) 


where the index w is the Hamming weight. When w = 1 the N single 
error bit patterns occur, w = N yields the pattern where every bit in 
the word is regenerated erroneously, while Qy,2 is the largest set of 
error bit patterns. For each error bit pattern there is a unique A-factor, 
giving 2% — 1 different A-factors. For example, the A-factor A; applies 
to a PCM word whose msb bit is erroneously regenerated, and is 
computed by determining the mean square value of the error (x; — x;1) 

over all quantized levels when this particular bit error sequence error 
"is present. The A-factors have a number of attractive properties. They 
depend only on the source conditions, and not on the channel. Specif- 
ically, for the particular number of bits regenerated erroneously in the 
recovered words and their position in these words, the A-factors 
depend only on the type of bit allocation, the companding law, and 
the PDF of x. The A-factors are unaffected by how the bit errors 
originated. Thus, once we have computed the A-factors for a given 
source, i.e., input signal and PCM encoder, we can use these factors 
to calculate the noise due to transmission errors for random channels, 
fading channels, different modulation schemes, various channel-cod- 
ing strategies, and so on. Armed with the A-factors we can determine 
the transmission error power, «2, for any channel that can be charac- 
terized by the error sequence probabilities p,, 1 = 1 to 2% — 1. 


II]. PERFORMANCE CRITERION 


The objective performance criterion to be used here is the overall 
signal-to-noise ratio (s/n), 

és E{x?} ao: 

2 


s/n => 2 2 — 
€g + €c + €a € 


(11) 


where ¢? is the average noise power [see eq. (4)] and E{x?} or o? is the 
average power of the input sequence. We usually express o? in decibels 
relative to the input level at the start of clipping, i.e., relative to a 
level magnitude of unity. 


IV. THE GAUSSIAN CHANNEL 


Consider the situation where the channel errors are independent 
such that any bit may be in error with probability P. The digital noise 
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power is given by eq. (9) and can be partitioned into noise power 
components, 


eae tet... + cd, (12) 


where ¢j, &, ---, ew are the noise powers due to 1-, 2-, --- , N-bit 
errors per word, respectively. The first component is 


N 
a = P, 2 Al, (13) 
=1 


where P, is the probability of a word having a single bit error, namely 
P, = P(l -— Py. (14) 


The number of A-factors associated with a particular number of bit 
errors per word depends on N. For example, when N = 8, there are 8, 
28, 56, 70, 56, 28, 8, and 1 A-factors required in the computation of ¢?, 
c3, e%, ef, ef, e&, €#, and ef, respectively. Thus the noise power compo- 
nents of ¢2 are 


N N+) 
e@=P, ¥ A, + Pp y, A; +--+ + PnAov-, (15) 
l=1 l=N+1 


and P,.) is the probability that (-) bits in the N-bit word are regener- 
ated erroneously. Rearranging eq. (15) as 


ly 


e@=P, » Ayw=1,2,---,N (16) 
[=1,41 
= P,Su, (17) 
where 
w-1 
l= > (7) (18) 
k=0 
and 
k=1 


and expressing P,, in terms of the average bit error probability P, we 
have 


e2 = P(1 — P)N18, + P21 — P)N?S, +... + PYSy 
N 
= y P*(1 — P)*’S,y. (20) 
w=1 


From eqs. (16) to (19) we observe that S,, is the sum of the A-factors 
associated with the Q,, error sequencies {e;} containing w ones. 
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4.1 Determination of the A-factors 


Equation (9) shows that the noise power due to transmission errors 
can be calculated with the aid of a set of A-factors. For the input 
sequence {x} with its PDF px(x), the A-factors can be determined 
using eq. (8), viz: 

2X1 


A; = X px(x:)6:(x; — x1)’, (21) 


and the sample x is quantized to y; if it approximately resides in the 
range from x; — (6;/2) to x; + (6;/2). Figure 3 displays both compressor 
and expandor characteristics, and we observe that when the quantized 
level y; is expanded, the level x; is recovered. Thus if x falls within 6,, 
it is recovered at the receiver as x;; = x;, assuming no transmission 
errors occur. If transmission errors are present, the expanded sample 
at the receiver is x;,. If we assume a large number of quantization 
levels, the probability density function px(x) has an approximately 
constant value of px(x;) over the interval 6;. Now the range of the 
compressor function f(x) is 2, and therefore the N quantized levels 
are spaced apart by 2/2". Consequently, the slope f’(x) of the compres- 
sion curve f(x) for an input amplitude x; is 


gi-N 


6: ° 





f'(x) = (22) 
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and we write eq. (21) as 
2N-1 -1 1-N 
pxlf~ (yi) 2 il -1 2 
A, = io Oo (23) 
i pao ee eee 

to ease the computation of the A-factors. The values of A; depend on 
the PDF of x, the compression characteristic, the bit assignment, and 
the number and location of the erroneous bits in the PCM word. Thus, 
we need to determine the A-factors for each PCM system at different 
input levels. 

In this discourse we will perform numerical calculations for the 
input signal having an exponential PDF, 





2s) ; (24) 


(x) = = ex (- 
Px = J2 p Oy 


although our results can be generalized for other PDFs. Unless it is 
explicitly stated we will assume binary folded PCM is employed, where 
the msb is the polarity of the quantized sample and the remaining bits 
represent its magnitude. The PDF px(x) of eq. (24) is assumed to be 
symmetrical about the zero voltage level, and the compandor is an odd 
function spanning the range from —1 to +1. 


4.2 Special case of single error A-factors and p-law companding 


To calculate the A-factors for a single bit error in any of the N-bit 
positions of a PCM word, we express eq. (21) as® 


Qn] 
A,= > px(xi)d:ai(x,), (25) 
i=0 
where 
a,(x;) = (x; = Xi1)"3 l = 1G 2, weg N, (26) 
and /= 1, 2, --- , N corresponds to a single error in the msb, the next 
msb, ---,]sb, respectively. Thus, the a;(x;) terms are related to a 


specific single bit error in the /th bit position, and a specific input 
signal amplitude x;. Observe that a;(x;) is independent of px(x;), and 
is a function of the PCM bit code and the compandor rule. By 
performing the summations in eq. (25) we obtain expressions for the 
single-error A-factors for a particular PDF px(x;). However, we are 
concerned here in how the effect of a particular single error varies 
with x;, and this we can determine with the aid of the a;(x;) functions. 
Of course, functions of the type of a;(x;) can also be derived for any 
particular multiple-error pattern e,. The analysis, however, gets in- 
creasingly complex. 

Before providing a general formula for a;(x;), we will consider for 
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exemplary purposes the case of an error occurring in the second bit 
position, i.e., ] = 2. Consider the input sample to have a value x, 
which is compressed to y, = f(x:). Because we are discussing the most 
significant magnitude bit, an error in this bit will either result in y, 
increasing by 1/2 when a logical 0 is regenerated as a logical 1, or 
decreasing by a 1/2 when a logical 1 is regenerated as a logical 0. The 
erroneous decoded y, is f-[f(x1) + (1/2)] or f-'[f(x1) — (1/2)], de- 
pending on whether the original bit was a logical 0 or 1, respectively. 
Thus for any x; we may express a2(x;) from eq. (26) as 


A2(x;) = alae Bs A ~ | ; 


This method of reasoning enables us to obtain a,(x;) for every value 
of 1. For a,(x;) we have 


ay(x;) = (2x;), 0< xis iF (27) 


and the general formula for a;(x;), 1 = 2 is 


{n° fx + aa - «i for f7 It 


= v vy odd 
cxf {ata y=1,.3.... 9U-)) 4 
a(x) = eT (28) 


fon -zaf-af me fs 


v yp even 
Pa = ee esi, OF 
S Xj < f {aeup y= 2, 4, ie: 9-1) 


So far we have not specified the compression function f(x;), but as 
it is our intention to subsequently place considerable emphasis on 
p-law PCM we provide now the a;(x;) terms for this type of compres- 
sion. The p-law compression function is specified by’ 


_ log(1 + ux;) 


and the corresponding expansion to recover <x; is 
2 1 
xi = f(y) oF fexp[ylog(1 + w)]-1j, O<y<1. (30) 


Equation (27) is applicable for u-law PCM, and the general formula 
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for! =2is 


{_1 ; 2 oe eee 
f =D (1 + px;) for f rn 


- v v odd 
<xj<f" | xen} p= 1, 3,5 209 1 
a(x;) = (31) 


: le 1 
FED pe -) JY 
Ie) alan ie Lae] (Leste): for: of ear 
z ee PEN py even 
<x<f ED [? y=24,--.,209, 


Inserting a;(x;) and px(x;) from eqs. (27), (81), and (24), respectively, 
into eq. (25) enables us to compute the single bit error A-factors for 
p-law PCM. 

The variation of the a;(x;) factors as a function of quantized 
level x;, / = 1, 2, --- ,8, is displayed in Fig. 4 for 8-bit u-law PCM, 
p= 255. As we expected from eq. (27), where / = 1, a;(x;) is a monotonic 
function as shown in Fig. 4a. Let us consider the case of ! = 2 when 
the single error occurs in the magnitude bit. To provide some insight 
into a2(x;), we note that its value in eq. (28) is 


1 : 1 
e eo + 4 _ x 5 O<x<f? (3 


reo = A _ sf - a (3) <x <1. 


If we substitute f(x;) into a2(x;), and for the moment confine ourselves 
to 0 < x; < f7! (1/2), we have 


Jay jlog(l + uxi) 1)... i 
a2 (x;) =| eles 7 aE 4 ab, (33) 


a(x) = 


and on applying eq. (30) 
1 1 : 
a2(x;) = |— \exp[log(1 + ux;) + 3 log rg) = Lea 
be 


= F (vi + i) (1 + px;)”. (34) 


m 
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Fig. 4—Variation of a;(x;) as a function of x; for 8-bit y-law PCM, » = 255. The 
subfigures a through h apply for a single bit error in the msb (J = 1), the next msb (J = 


2), --+,Isb (J = 8), respectively. 


From eq. (30) the value of x; when y; = 1/2 is 


cs (3) = i [elt/2}log(i+u) _ 1] 
Le 


== (WT Fu - 2), 


and hence we obtain 


(35) 


ay(x;) = I (3) (1 + wx;)?; 0<x,<f" (3), (36) 
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Fig. 4—Continued. 


the value given by eq. (31). Observe that y; = 1/2 marks the boundary 
between the most significant magnitude bit being 0 (y; < 1/2), and 
being a logical 1 (y; = 1/2). Consequently, an error in the most 
significant magnitude bit changes the magnitude of the quantized 
signal by a 1/2. However, the change in x; is not a 1/2, being dependent 
on the p-law. Figure 5 illustrates that if the input sample has a small 
amplitude, x; say, and is compressed to ¥,, then the effect of an error 
in the most significant magnitude bit is to cause y, to increase by half 
the range to yi. The corresponding decoded signal is x;, having an 
error E, of Vao(x;). If the input sample amplitude is larger, say x2, 
and the same bit is inverted, x2; is recovered having an error KE» of 
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Fig. 4—Continued. 


Vd2(x2) that is significantly larger than E,. Thus the errors become 
significantly larger as y; moves closer to 1/2. The variation of a2(x;) over 
the range of x from zero to f~' (1/2) is shown in Fig. 4b. As we ex- 
pected from eq. (36), when x; is very small, a2(x;) is a constant equal to 
[f-'(1/2) }?. As x; approaches {71(1/2), 


a2(x;) = [f-1(1/2)Pu?x? (37) 


and the variation appears as a straight line in the figure. 

Consider the situation when x; exceeds f~'(1/2). An input x3 (see 
Fig. 5), encoded to y3 and then subjected to an error in the second bit 
position, is reduced to x3, and the resulting error E3 of Va2(x3) is less 
than E, and of opposite polarity. We therefore observe in Fig. 4b a 
discontinuity in a2(x;) as x; just exceeds f~'(1/2), causing it to plum- 
ment. By applying similar arguments to those employed in establishing 
eq. (36), we obtain the result given in eq. (31), namely, 


cng yl 
IG) , 
a2(x;) = 14h (see of (2) <x <1. (38) 


When px >> 1, ao(x;) is proportional to x?, as can be seen in Fig. 4b 
for x; = ‘ae (1/2). 
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When errors occur in the third msb, i.e., ] = 3, we have from eq. 


(31), 
—1 1 2 1 z “)?s <y< is i 
4 ( UX; > = xi 4 


1 1 
ore (D+ ux)’ f7 (7 <xj<f" (3) 
a3(x;) = (39) 


i(lfornn Glance 
Ole pl 


eT cee eee, .\2. “lfoje< < —1 , 
V1 +p Ch ee) f (=) <« f (1) 


From this equation the variation of a3(x;) with x; is evident. The 
abrupt changes occur at f~1(1/4), f-'(1/2), and f-1(3/4), because the 
transmission errors cause y; to change by 1/4. Thus the detailed 
discussion relating to a2(x;) and Fig. 5 are relevant here, but instead 
of one transition at f~'(1/2) there are three logical transitions for this 
case of / = 3. 

For higher values of / there are more transitions in y; due to 
transmission errors, and these manifest as more discontinuities in 
a,(x;). For l= 8 the logical value of the lsb changes with each succeeding 
x;, and as a consequence there are 128 discontinuities in ag(x;), al- 
though the magnitude of the jumps in ag(x;) at the discontinuities are 
miniscule. The result is that the curve of ag(x;), shown in Fig. 4h, 
appears to be almost smooth. 


cn) 


4.3 Computing the overall s/n 


The digital noise power «2 is dependent on the type of digital 
modulation. For noncoherent frequency shift keying (NCFSK) the bit 
error probability is 


E 


Pare ®, (40) 





where F jis the bit energy and N, is the one-sided spectral density 
function of the white Gaussian noise in the channel. When coherent 
phase shift keying (CPSK) is employed, the bit error probability is 
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Fig. 5—The nonlinear quantizer characteristic showing how the effect of a transmis- 
sion error in the second msb (1 = 2) affects the amplitude of the recovered sample. 


given by’? 
2E 
po \/) 7 


When the values of P given by eq. (40) or eq. (41) are substituted into 
eq. (20), the digital noise power for NCFSK and CPSK are determined. 
The knowledge of ¢2 enables s/n to be determined using eq. (11). 


V. RAYLEIGH FADING CHANNELS 


In Section IV we provide an expression [see eq. (20)] that enables 
us to calculate the noise power in the recovered signal due to trans- 
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mission errors in p-law PCM. The bit errors are considered to be 
statistically independent. We now turn our attention to transmission 
channels that contain both additive Gaussian noise and Rayleigh 
signal fading. Our investigation considers both the effects of scram- 
bling (also called interleaving) and not scrambling the bits prior to 
transmission, as well as the application of diversity techniques to 
reduce the digital noise power. We commence by considering the case 
of NCFSK modulation. 


5.1 Digital noise power with NCFSK modulation 


5.1.1 Bit scrambling prior to transmission 


If the bit stream is scrambled prior to its transmission over a 
Rayleigh fading channel, and the regenerated bit stream is descram- 
bled at the receiver, then the bit errors will tend to be statistically 
independent. By the process of scrambling, the burst errors that occur 
in Rayleigh fading environments are randomized. Because the trans- 
mitted signal is subjected to Rayleigh fading, the channel s/n y is a 
random variable having a PDF of’? 


1 
foece™, (42) 
r 
where I is the average bit signal-to-noise ratio, viz: 


r= Efy}. (43) 


The average bit error probability for the Rayleigh fading channel with 
its additive Gaussian noise is’® 


P= i: f(y) P(y)dy, (44) 
where 
he 2 
P(y) = 9° (45) 
resulting in 
Pr= a (46) 
PO o4T 


Substituting Pr from eq. (46) as P into eq. (20) yields the digital noise 
power for bit scrambling, NCFSK, and a Rayleigh fading channel, viz: 


N 1 w 1 N-w 
2 eg 1 = Sw 
cans = 2 (; + ;) ( 2+ =) 
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or €2 7s may be expressed as 


N Je N 
2 = Re aL eee ae Ww 


where the subscripts F and S signify a fading channel and the appli- 
cation of bit scrambling, respectively. The term T,, is 


fie = > Dis & a) (-1)"7; w= Lj 2, eeuas N, (48) 
j=l w— J 


and S,, can be determined from eqs. (16) through (19). For Pr « 1, 
the only value of w that need be considered is 1, and hence 
€2 ~ Ty ace Sy 
oT 24Tr 24Tr 








(49) 


5.1.2 No bit scrambling prior to transmission 


We assume that the fading is sufficiently slow for y to be constant 
over an N-bit period, i.e., over the duration of one PCM word. The 
probability of w bits being erroneously regenerated for a particular 
value of y is P(y)“[1 — P(y)]*~”. When the bits are not scrambled the 
average probability of w bits being regenerated erroneously is 


Pues = f P(y)*{1 — Ply)" fly)dy, (50) 
where the subscripts w, F, S, signify w bits in error, a fading situation, 
and no bit scrambling, respectively. Applying a similar procedure to 
that used in establishing eq. (20) enables us to determine the digital 


noise power for the Rayleigh fading channel when no bit scrambling 
is employed, viz: 


CFS = S1 f P(y)[L — Ply) "Ff v)dy 
+ Se f Ply)" — Pry *f(y)dy 


pas $ sf P(y)"f(y)dy 
N 
= ¥ ParsSw. (51) 


An alternative representation is 
N eo 
cars= L. Tw ‘ Fly) P* (yay, (52) 
w=1 o 
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where 7, is given by eq. (48). The integral expression in eq. (52) is, 
from eqs. (42) and (45), 


1 


f f(y) P* (y)dy = 2-1 + Pw)’ 


(53) 
and consequently the digital noise power for a slow fading channel 
and no bit scrambling is 

N T., 


enRS > > 


war 2"-1(2 + Tw) ee 


5.1.3 Effect of diversity 


The average digital noise power formulae will now be derived when 
diversity is applied, where the type of diversity, e.g., space, frequency, 
time, polarization, is immaterial. For M-fold diversity reception with 
ideal maximal-ratio combining, the PDF of the channel s/n y is®® 


7 aaa a 


fy = PiM— pr (55) 
If the receiver selects the diversity branch having the strongest signal 
level the diversity scheme is known as ideal selection combining for 


which the PDF is 


fly) = eM = eye, (56) 


5.1.3.1 Bit scrambling employed. Substituting f(y) from eq. (55) and 
P(y) from eq. (45) into eq. (44) gives the average bit error probability. 
When this probability is substituted in eq. (47) the digital noise for 
the ideal maximal-ratio i aes becomes 


ace Nd 
a SE ar | 57 
where the subscript D signifies the application of diversity reception. 

When the process is repeated using f(y) from eq. (56), the digital 
noise power for ideal selection combining is obtained, viz: 


w 


ae a 
: = f a 1)’ ————— | . (58 
€3,F.S,D 2 or 2 (-1) F r : F : i (58) 
rr fF 2 


5.1.3.2 No bit scrambling prior to transmission. When ideal maximal- 
ratio combining is employed, application of eqs. (45) and (55) yields 

9M—w 
(2+ Tw)” 


i P*(y)f(y)dy = (59) 
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for NCFSK modulation. The digital noise power is then determined 
to be 


QM—w 
- 2 Most ©) 
For ideal selection combining the expression 
7 M *™ = | 1 
if P*(yi(ydy = saa y (— ~1y (M laa (61) 
oO T2 J 1 at J ica 
Poot. 2 


is established by means of eqs. (45) and (56) and the application of 
the technique described in Ref. 14. The digital noise power for the 
situation described here is 


a N MT ‘ ros | 1 
a = 2X Tor T2” Zn (M j [eeene e2) 
TT 2 


5.2 Digital noise power with CPSK modulation 


The differences in the digital noise power of the PCM system using 
CPSK and NCFSK modulation are due to the differences of their 
probability of bit error. For CPSK the probability of bit error as a 
function of channel s/n is*® 


P(y) = Q(v2y). (63) 


The distribution functions for the channel s/n’s are given by eqs. (55) 
and (56) for the case of maximal-ratio-combining diversity and ideal 
selection-combining diversity, respectively. When bit scrambling is 
performed prior to transmission, the average bit error probability is 
computed using eq. (44), where P(y) is given by eq. (63), and f(y) is 
selected from either eq. (55) or (56). This average bit error probability 
is substituted into eq. (47) to give the digital noise power, from which, 
with the aid of eq. (11) s/n is formulated. 

When the bits are not scrambled, the digital noise power is found 
by substituting P(y) from eq. (63), and f(y) from either eq. (55) or 
(56) directly into eq. (52). With the aid of eqs. (52) and (11), s/n is 
determined. 


VI. MINIMUM DISTANCE CODE PCM 


The folded binary PCM code has its msb as the polarity of the 
encoded sample, while the remaining bits constitute the magnitude of 
the sample. In natural binary code the most negative sample is given 
an all-zero pattern, and binary numbering of progressively more pos- 


974 TECHNICAL JOURNAL, JULY-AUGUST 1984 


itive samples ensues until the most positive sample is reached, which 
is assigned an all-one code. A single error in these codes can produce 
large errors in the recovered samples. The Gray code is also vulnerable 
to transmission errors, although the magnitude of the errors is not as 
large as those encountered when the folded binary and natural binary 
codes are employed. 

The Minimum Distance Code (MDC) was conceived by Rydbeck 
and Sundberg.* The word distance in this code name refers to the 
difference between a recovered sample correctly received and its value 
when subjected to a single bit error. The MDC has the property of 
asymptotic minimum digital noise for low-level speech samples, and 
the abbreviation may also be taken as minimum digital noise code. 
The structure of the code is based on the error sequencies discussed 
in Section II. The msb represents the polarity of the quantized sample, 
being logical 0 for positive samples and logical 1 for negative samples. 
Table I shows the seven-bit magnitude code for 8-bit PCM. For 
quantized level number y; of zero we have an all-zero word. The code 
words for y; = 1 to 7 are the single error patterns when the Hamming 
weight w is unity. The next group is the double error (w = 2) patterns 
having 21 components extending from y; = 8 to 28 inclusive. The code 
progresses, having 35 triple-error sequencies, w = 3, and so on until 
the final all logical 1 code when w = 7. The number code words in 
each group represented by the Hamming weight w are 


se) 


The MDC code is specifically designed to accommodate low-level 
input signals. When single bit errors occur the code can only change 
to an adjacent group. Changes involving half the magnitude range that 
occurs in folded binary PCM are forbidden. However, errors in the 
least significant bits will in general cause much larger errors in the 
recovered sample than in folded binary. We therefore trade the large 
erroneous spikes in the recovered signal that occur in folded binary 
PCM for smaller spikes that happen more frequently in MDC. 


Vil. PERFORMANCE 


The input range of the quantizer was from —1 to +1, and in our 
theoretical calculations the input signal was assumed to have an 
exponential PDF with standard deviation of either /2/10 (—17 dB), 
or 0.01 (—40 dB). The 8-bit u-law PCM encoder, uv = 255, was arranged 
to operate at input signal powers of —17 dB and —40 dB, the former 
being indicative of high-level speech and the latter of low-level speech. 
Three binary codes were considered initially—Natural Binary Code 
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Table |—The MDC code for 8-bit PCM (the polarity bit is not shown) 
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Table |I—Continued 











Quantized Quantized 
Level Magnitude Level Magnitude 
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(NBC), Folded Binary Code (FBC), and Minimum Distance Code 
(MDC)—but it soon transpired that NBC was considerably inferior 
to the others and was abandoned. In Table II we display the single- 
error A-factors and the T-factors of eqs. (25) and (48), respectively. 
The input signal has an exponential PDF specified by eq. (24), and 
the codec is 8-bit u-law PCM, pw = 255. The factors A;, Ao, --- , Ag are 
associated with single errors in the first, second, --- ,lsb, respec- 
tively. The entries in the table illustrate that the variation between 
the A-factors is significantly smaller for MDC than for FBC. Conse- 
quently, the digital noise power due to single bit errors per word when 
the MDC code is used is similar for all bit locations in the words. This 
situation is radically different when FBC is employed. For low-level 
speech the factor T, for MDC is significantly smaller than its coun- 
terpart with FBC. When the input level is high, the 7) factors are 
approximately the same. Unlike the case of linear PCM with FBC, 
where 7; to Ts are zero,’® the T-factors with companded PCM are 
nonzero. 

The types of modulation employed in our analysis were NCFSK and 
CPSK. The quantization noise power «7 and clipping noise power ¢2 
were determined for the input powers of —17 and —40 dB in the 
absence of transmission errors. The digital noise power «2 was com- 
puted using the equations in Sections IV and V. Armed with knowledge 
of these noise powers we were able to compute the theoretical overall 
s/n, namely s/n, as expressed by eq. (11). As €2 is a function of channel 
s/n, the variation of s/n with channel s/n was determined. 

In addition to presenting these theoretical variations of s/n, we also 
conducted simulations using speech signals. In our experiments we 
used four concatenated sentences: “Glue the sheet to the dark blue 
background,” “Rice is often served in round bowls,” “Four hours of 
steady work faced us,” and “The box was thrown beside the parked 
truck.” The first two sentences were spoken by females, the others by 
males. The speech was bandlimited between 200 and 3200 Hz, and 
sampled at 8 kHz to give the speech sequence used in our simulations. 
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Table II—The single error A- and T-factors for 8-bit u-law PCM, » = 255; the input PDF is exponential 


Ai 


—40 dB/FBC 3.954e-4 
-—40 dB/MDC 3.954e-4 
-17 dB/FBC 0.07906 
~17 dB/MDC 0.07906 


T; 


—40dB/FBC —_ 0.03900 
~40 dB/MDC 0.01060 
-17dB/FBC 0.3223 
-17dB/MDC_ 0.3352 


Az 


3.783e-2 
2.486e-3 
0.1391 
0.05214 


Te 


0.2871 

0.3019 
—0.1182 
—0.06155 


As A, 
6.375e-4 1.105e-4 
2.220e-3 1.982e-3 
0.08739 0.01331 
0.04879 0.04554 

T3 T4 
4.914e-3 —4,108e-2 
0.3811 ~—1.024 

—0.2133 0.06660 
—0.7784 1.480 


As 


2.184e-5 
1.632e-3 
2.644e-3 
0.04118 


Ts 


2.155e-4 

0.9891 

0.003221 
—1.660 


Ag 


5.176e-6 
1.161e-3 
6.281e-4 
0.3434 


Te 


6.056e-5 
—0.4009 
—1.976e-4 

1.052 


Ay 


1.277e-6 
5.833e-4 
1.550e-4 
0.02405 


T; 


3.576e-7 

0.03743 

2.980e-7 
—0.4118 


As 


3.181e-7 
1.389e-4 
3.863e-5 
0.01009 


Ts 


~—1.788e-7 

—4,965e-3 

—1.192e-7 
0.1041 
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Fig. 6—Gaussian channel. Theoretical curves of overall s/n against channel s/n for 


NCFSK modulation (a) —17 dB input power, MDC; (b) —17 dB input power, FBC; (c) 
~—40 dB input power, MDC; and (d) —40 dB input power, FBC. 


7.1 Gaussian channel 


The theoretical curves of s/n versus channel s/n when NCFSK 
modulation was employed are displayed in Fig. 6. When the input 
signal power was —17 dB there was negligible difference between the 
curves corresponding to FBC and MDC binary codes. Reducing the 
input signal power to —40 dB revealed differences in the two codes, 
with MDC producing a gain in s/n compared to FBC of up to 4 dB for 
a given channel s/n. 

The theoretical results for CPSK, shown in Fig. 7, are essentially 
the same as for NCFSK, except that the curves are shifted by approx- 
imately 4 dB to lower values of channel s/n. When speech was used 
in the simulation, the modulation being CPSK, the results are those 
shown in Fig. 8. The curves in this figure were shifted by approximately 
2.5 dB to higher values of channel s/n compared to the theoretical 
curves. The deterioration in performance compared to the theoretical 
curves occurs because of the diverse nature of speech with its voiced 
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Fig. 7—Gaussian channel. Theoretical curves of overall s/n against channel s/n for 
cps modulation (a) —17 dB input power, MDC; (b) —17 dB input power, FBC; (c) 
-~40 dB input power, MDC; and (d) —40 dB input power, FBC. 


and unvoiced sections, and its intraword and interword silences. The 
theoretical curves apply to input signals having exponential PDFs, 
while speech is notorious for its nonstationary statistics. Nevertheless, 
sources having exponential PDFs are often used to represent speech, 
and clearly the theoretical and speech curves do have similar shapes, 
albeit their difference in absolute s/n for a given channel s/n. This 
curve similarity is particularly evident when we compare the perform- 
ances of FBC and MDC at input levels of —17 and —40 dB. 


7.2 Rayleigh fading channel 


Figure 9 shows the variation of s/n as a function of channel s/n for 
a Rayleigh fading channel and for CPSK modulation. The theoretical 
curves in this figure relate to a number of conditions. For each of the 
two input power levels of —17 dB and —40 dB, we computed the curves 
for FBC and MDC when the bits were scrambled prior to transmission 
and when no scrambling was performed. The four curves, a, b, c, and 
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Fig. 8—Gaussian channel. Speech simulation curves of overall s/n against channel 
s/n for CPSK modulation (a) —17 dB input power, MDC; (b) —17 dB input power, FBC; 
(c) —40 dB input power, MDC; and (d) —40 dB input power, FBC. 


d, coaleased for the input power of —17 dB. For a channel s/n of 30 
dB, and the lower input power, MDC had a s/n advantage of 5 dB 
over FBC when bit scrambling was performed, and only a 0.8-dB gain 
in s/n in the absence of bit scrambling. We observed that MDC with 
bit scrambling, curve e, had a gain in s/n of 9 dB compared to curve 
h, where FBC without bit scrambling was employed. These gains in 
s/n are valid for channel s/n values where s/n > 0 dB. 

We also computed the theoretical curves for NCFSK, and we ob- 
tained curves that were almost identical to those shown in Fig. 9 but 
shifted by 6 dB to higher values of channel s/n. As with CPSK, over 
most of the range of channel s/n the curves are parallel, indicating 
that as the Bit Error Rate (BER) increases the relative performance 
of the different codes is unchanged. 


7.2.1 PCM speech over mobile radio channels 
The u-law PCM encoded speech was allowed to two-level CPSK 
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Fig. 9—Rayleigh fading channel. Theoretical curves of overall s/n against channel 
s/n for nondiversity CPSK modulation: (a) -17 dB input power, bit scrambling, MDC; 
(b) —17 dB input power, bit scrambling, FBC; (c) —17 dB input power, no bit scrambling, 
MDC; (d) —17 dB input power, no bit scrambling, FBC; (e) —40 dB input power, bit 
scrambling, MDC; (f) —40 dB input power, bit scrambling, FBC; (g) —40 dB input 
power, no bit scrambling, MDC; and (h) —40 dB input power, no bit scrambling, FBC. 


modulate an RF carrier. A sequence of the envelope function A(t) was 
generated from a hardware simulator of frequency-selective Rayleigh- 
fading mobile radio paths.'” The A(t) signal was sampled at 32 kHz 
and inserted into the computer to provide the fading envelope of a 
vehicle traveling at 15 mph. By resampling A(t) we were able to 
produce fading envelopes for different vehicular speeds. The CPSK 
signal was subjected to these fading envelopes and corrupted by 
cochannel interference. The recovered baseband signal was sampled 
at the PCM bit rate to produce at the kth instant,’” 


B, = R,b; + Th, (64) 


where R,, b,, and J; were the amplitudes of the Rayleigh envelope, 
transmitted bit, and additive interference level, respectively. The 
regenerated bit, b,, was a logical one if B, = 0; otherwise it was a 
logical zero. The bits were grouped into PCM words and decoded using 
u-law PCM. The difference between the input speech sequence and 
the decoded PCM sequence yielded the error sequence. The over s/n 
was then computed. 

Figure 10 shows the variation of the s/n for our speech signal as a 
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Fig. 10—Mobile radio channel. Speech simulation curves of overall s/n against 
channel s/n for nondiversity CPSK modulation with a vehicular speed of 30 mph. Curve 
labels are as in Fig. 9. 


function of the channel s/n, for a vehicular speed of 30 mph. Eight 
curves are displayed corresponding to those used in Fig. 9. The curves 
of Fig. 10 have the same general shape as those of Fig. 9. We repeated 
our experiment for vehicular speeds of 60 mph, and overlaid the curves 
for this speed with those shown in Fig. 10. The fit for the high input 
levels was fairly exact, but for the —40 dB input level there were 
substantial discrepancies. Had we been able to take sufficiently long 
segments of speech and channel variations, our curves at —40 dB 
would have approached those in Fig. 9, subject to the inherent dis- 
crepancy that speech does not have the exponential distribution as- 
sumed for the theoretical case. The greater variations in the slopes of 
the curves of Fig. 10 compared to those in Fig. 9 at the —40 dB input 
level compared to the —17 dB level is to be expected, since for the 
same error rate the effect of an error can have a much greater influence 
on s/n when the input level is low. 

We display in Fig. 11 a segment of the original input speech 
waveform, and the corresponding recovered waveforms obtained for 
conditions marked A and B in Fig. 10, respectively. The characteristic 
large error spikes associated with FBC are evident in Fig. 11b, where 
a spike can exceed half the amplitude range of the input speech signal. 
The nature of the MDC is to cause small spikes when the input speech 
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Fig. 11—y-law PCM speech waveforms for mobile radio channels, nondiversity 
CPSK, vehicular speed 30 mph, —40 dB input power level, 25-dB channel s/n, and no 
bit scrambling: (a) original speech; (b) recovered speech with FBC (point A in Fig. 10); 
and (c) recovered speech with MDC (point B in Fig. 10). 


samples are small, or indeed, when they are very large (see Table I 
and Fig. 11c). Errors in samples quantized to the middle of the range 
are responsible for the largest spikes with MDC. 


7.2.2 Effect of diversity 


When 2-branch (M = 2) diversity is applied, the value of s/n depends 
upon whether we use ideal maximum-ratio-combining diversity 
(IMRCD), or ideal selection-combining diversity (ISCD). Figure 12 
displays the variation of s/n against channel s/n for numerous condi- 
tions, the modulation being CPSK. We observe that IMRCD is pref- 
erable to ISCD, that MDC coupled with bit scrambling is the best of 
our choices at the —40 dB input level, and that the curves have the 
same form as for M = 1 (see Fig. 9), except that the system can now 
operate at significantly lower values of channel s/n. 

We do not show the theoretical curves for NCFSK for M = 2. Suffice 
it to say that they are virtually identical to those of CPSK, M = 2, 
except for a shift of 6 dB to higher values of channel s/n. 

For the mobile radio channel described in Section 7.2.1 the curves 
of Fig. 13 were obtained. For reasons of clarity, we show only those 
curves with bit scrambling. These results are similar to those in Fig. 
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Fig. 12—Rayleigh fading channel (2-branch diversity). Theoretical curves of overall 
s/n against channel s/n for CPSK modulation: (a) —17 dB input power, IMRCD, bit 
scrambling and no bit scrambling, FBC, and MDC; (b) —17 dB input power, ISCD, bit 
scrambling and no bit scrambling, FBC, and MDC; (c) —40 dB input power, IMRCD, 
bit scrambling, MDC; (d) —40 dB input power, ISCD, bit scrambling, MDC; (e) -40 dB 
input power, IMRCD, bit scrambling, FBC, also no bit scrambling, MDC; (f) —40 dB 
input power, IMRCD, no bit scrambling, FBC; (g) —40 dB input power, ISCD, bit 
scrambling, FBC, also no bit scrambling, MDC; and (h) —40 dB input power, ISCD, no 
bit scrambling, FBC. 


12, where the somewhat erratic behavior of the curves at the lower 
input power level is attributable to the reasons discussed in connection 
with the corresponding curves in Fig. 10. 

The effect of using different values of M on the theoretical s/n is 
displayed in Fig. 14 for NCFSK, an input power level of —17 dB, 
MDC, bit scrambling, and IMRCD and ISCD. As M is increased, the 
knee of the curves moves to lower values of channel s/n, and the 
advantage of using IMRCD over ISCD increases. Figure 15 shows the 
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Fig. 13—Mobile radio channel. Speech simulation curves of overall s/n against 
channel s/n for CPSK modulation. The vehicular speed is 30 mph, with 2-branch 
diversity and bit scrambling employed: (a) —17 dB input power, IMRCD, MDC; (b) —17 
dB input power, IMRCD, FBC; (c) —17 dB input power, ISCD, MDC; (d) —17 dB input 
power, ISCD, FBC; (e) —40 dB input power, IMRCD, MDC; (f) —40 dB input power, 
eae a (g) —40 dB input power, ISCD, MDC; and (h) —40 dB input power, 


theoretical curves for the same conditions using CPSK instead of 
NCFSK. Again we note the close correspondence between the curves 
if a lateral shift in channel s/n is made. 


VII. DISCUSSION 


Digital noise in u-law PCM systems has been examined for both 
Gaussian and Rayleigh fading channels. Theoretical results for the 
digital noise power, i.e., the noise power in the decoded signal, have 
been presented for both NCFSK and CPSK modulation. Underpinning 
this theory is the application of the A-factors, and accordingly we have 
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Fig. 14—Effect of diversity on Rayleigh fading channels. Theoretical cuves of overall 
s/n against channel s/n for -17 input power, NCFSK modulation, bit scrambling: 
(a) no diversity; (b) ISCD (M = 2); (c) IMRCD (M = 2); (d) ISCD (M = 4); and (e) 
IMRCD (M = 4). 


elucidated their significance in detail. Armed with knowledge of the 
A-factors the determination of the digital noise power is greatly 
simplified [see eq. (9)]. We emphasize that although we have concen- 
trated our attention on p-law PCM, the A-factor approach is equally 
valid for A-law PCM. For this case the A-factors are computed by 
replacing the p-law eqs. (29) and (30) by the A-law equations,"* and 
proceeding as before. 

Two binary codes for the u-law PCM words were investigated: folded 
binary code (FBC) and minimum distance code (MDC). The MDC 
code yielded significant gains in s/n compared to FBC, with the proviso 
that the input power level was low. This is readily apparent upon 
inspection of Figs. 6 through 13. In our deliberations on Rayleigh 
fading channels we considered the effect of scrambling the bits in the 
PCM sequence prior to transmission, and compared it to the situation 
when bit scrambling was not employed. At low input power levels bit 
scrambling produced a gain of 8 dB when MDC was used, as shown in 
Fig. 9. The application of diversity made a significant contribution to 
system performance, and our best results occurred when we used a 
combination of ideal maximal-ratio-combining diversity, bit scram- 
bling, and MDC binary code. 
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Fig. 15—Effect of diversity on Rayleigh fading channels. Same conditions as in Fig. 
14, except that CPSK modulation is used. 


Four concatenated sentences of speech were used in our simulations, 
and we have shown in our figures that there is consistency in the 
shape of the simulated and theoretical curves. In particular, at the 
higher input level the simulated and theoretical s/n values agree to 
within a couple of decibels, but for second-order diversity the discrep- 
ancy is = 4 dB. These departures from the theory are a consequence 
of deriving our formulae for an input signal having an exponential 
PDF. Speech, by contrast, has time-varying statistics. However, voiced 
speech constitutes nearly 80 percent of speech, and its PDF can be 
approximated by an exponential one. It is for this reason that we used 
the exponential PDF in our theoretical deliberations. 
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and D, in Silica Optical Fibers 
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We have recorded quantitative optical absorption and Raman spectra of Hz 
and Dz, diffused into silica optical fibers at a pressure of 500 atm. The 
absorption spectra recorded were fundamental through the third overtone for 
H, (0.6 through 2.5 wm) and the first through fourth overtone for D, (0.7 
through 1.7 pm). The observed spectra show fine structure similar to what has 
been seen in gas-phase Raman spectra. By considering all our absorption and 
Raman spectra for both H, and D,, including temperature dependence and 
anharmonicities, and comparing the spectra to gas-phase Raman spectra, we 
have made unambiguous assignments for all observed spectral lines. As a 
result of these assignments, we conclude that H, and Dz are trapped singly in 
interstitial sites in silica and are free to rotate. The absorption spectra provide 
information on the added loss due to Hz saturation of silica optical fibers. For 
example, extrapolating the experimental results to the case of a fiber saturated 
under 1 atm of Hz, the added loss due to the intense first overtone Q-branch 
line is 14 dB/km at 1.2445 um and 0.14 dB/km at 1.30 um. High loss due to 
the fundamental absorption of H2 occurs beyond 1.5 um; at 1.55 um it is 0.8 
dB/km for 1 atm of He. There is, however, a transmission window in H- 
saturated fibers between 1.32 and 1.45 um. 


I. INTRODUCTION 


During a study of the Raman gain properties of molecular hydrogen 
(H,) diffused into silica fiber, it was found that Hg in silica is infrared 
active with a broad absorption spectrum in a wavelength region 
extending from 2.4 um into the visible. The report of this work showed 
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a portion of the first overtone spectrum between 1.0 and 1.4 um (a 
region of interest for fiber optical transmission), along with tentative 
spectroscopic assignments.’ This He optical absorption in fibers sub- 
sequently was found to be of practical importance to lightwave com- 
munications with the observation of in situ generation of He in optical 
fiber cables due to electrolysis.2* Also, the recent literature contains 
calibrated first-overtone loss spectra for silica fibers containing He 
under about 1-atm pressure, as well as spectral assignments for the 
fundamental and first-overtone regions.'*> We show here that all of 
the reported assignments are at least partially incorrect. 

To obtain sufficient information for unambiguous assignment of all 
observed absorption lines, we have observed the absorption spectra of 
silica fibers impregnated with Hz or De at pressures that (1) are 
sufficiently high (500 atm) to produce gas concentrations in the fibers 
close to their saturation limits in silica, and (2) are larger by a factor 
of 50 than the highest pressures reported in other recent experiments.* 
The observations include calibrated fundamental and first- through 
third-overtone spectra for Hg, first- through fourth-overtone spectra 
for Dz, and spontaneous Raman spectra for both Hz and Dz. The 
assignments made possible by measurements of these spectra also 
indicate that the “guest” molecules are trapped singly in the silica and 
are free to rotate. 


Il. EXPERIMENTS 


In the experiments we employed several-hundred-meter lengths of 
multimode silica fiber with a 50-um core and 125-ym outer diameter. 
The core contained a graded concentration of GeO, (about 14 percent 
GeO, at the center) and a small amount of P.O;. One fiber was 
subjected to Hz at 500-atm pressure at room temperature for 30 days. 
We calculated the concentration of dissolved He using the Langmuir 
adsorption model, with data adjusted to 25°C.” The resulting concen- 
tration of H, in the fiber was 5.6 < 107° molecules/cm?, which is about 
500 times the reported concentration at 1 atm. We treated similar 
fiber with D, in the same way. In addition, we treated a 10m length of 
single-mode fiber in Hz under the same conditions for five days. After 
treatment the fiber was removed from the pressure vessel and its 
spectra were measured at ambient atmospheric pressure. Between 
measurements the fibers were stored in liquid nitrogen to eliminate 
out-diffusion. The absorption spectra of all samples were obtained 
using a tungsten-lamp source and a 3/4-meter monochromator; all 
were normalized to the spectrum of an equal length of untreated fiber, 
and all were fully resolved. We chose absorption path lengths to ensure 
that the measured absorptions were within the dynamic range of the 
instrumentation. Raman spectra were obtained on the same mono- 
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chromator with a krypton-ion laser at 647.1-nm as the excitation 
source. The spectra were taken at room temperature and 77K. 


Hl. RESULTS AND DISCUSSION 


In the gas phase, direct absorption of infrared radiation is a rigor- 
ously forbidden process for homonuclear diatomic molecules, such as 
H, and Dz, due to their lack of a transition dipole moment.’ The high 
symmetry of Hz and Dz dissolved in silica is reduced by the silica 
matrix surrounding the dissolved molecule, leading to induction of a 
transition dipole moment and absorption of infrared radiation reso- 
nant with molecular transition frequencies. Figure 1a shows the mea- 
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Fig. 1—First-overtone loss spectrum at room temperature of He in silica fiber: 

(a) Entire spectrum of the first overtone, measured in a 10m single-mode fiber; », and 


v3 are silica vibrations;® (b) Expanded view of the most intense band, the Q branch, 
measured on 2.3m of multimode fiber. 
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sured H, loss spectrum between 1.0 and 1.6 wm for the single-mode 
fiber. Figure 1b shows the main absorption band, i.e., the Q branch of 
the first-overtone transition, measured at high resolution in the mul- 
timode fiber. Figures 2 and 3 show the second and third He-overtone 
absorption regions, respectively, for the multimode fiber. We also 
recorded spectra for the fundamental absorption region of Hz and the 
first- through fourth-overtone regions of D2, as well as Raman spectra 
for H, and D2. HD was seen as an impurity in the Dz spectra. 

The spectra shown in Figs. 1 through 3 have the general features 
observed in all the absorption spectra. The strongest feature is iden- 
tified as the Q-branch transition. Table I shows transition energies, 
wavelengths, and attenuation coefficients (measured for 500-atm 
treatment pressure, and reduced to 1 atm) for the observed Q branches 
of Hy, Dz, and HD. The spacing between successive vibrational levels 
(Q-branch spacing) decreases due to anharmonicity. Table I also shows 
the measured high-pressure attenuation coefficients and the derived 
l-atm values. Note the large ratios of the Hz and Dz absorption 
strengths. We observed other weaker lines, and assigned these to 
vibration-rotation transitions, S(J) and O(J) lines (for a general de- 
scription of the Hy spectral lines see Ref. 8), or to combination 
absorptions with the silica tetrahedral modes »; (800 cm™') and ps 
(1100 cm7).° Figure 4 shows a correlation diagram of the Raman, 
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Fig. 2—Second-overtone spectrum of Hy at room temperature measured on 13m of 
multimode fiber. 
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Fig. 3—Third-overtone spectrum of Hz at room temperature measured on 450m of 
multimode fiber. 


fundamental, and overtone lines for Hg, all relative to the Q-branch 
lines. If we reference the observed lines to the appropriate Q-branch 
transition, the consistency of the fine structure in each vibrational 
transition becomes apparent. This procedure also eliminates the en- 
ergy shifts due to vibrational anharmonicity. Figure 5 shows similar 
results for the Raman and first four overtone lines of Do, as well as 
gas-phase Raman lines’® and the positions of the silica tetrahedral 
modes.? 

The assignment of the S(J) and O(J) vibration-rotation lines is 
based on three factors: 

1. The closer spacing and greater number of these lines in the D» 
spectra compared to the He spectra, in accord with general expecta- 
tions for the H-D isotope shifts.® 

2. The excellent quantitative agreement between the rotational 
spacings observed in the fiber spectra for H, and D,. compared to the 
spacings in the corresponding gas-phase spectra. 

3. The relative intensities of the lines and the variation of relative 
intensities with temperature. [At 295K, the most intense rotational 
lines are S(1) for He and S(2) for De. At 77K, the most intense lines 
are S(0) and S(1) for Hz and S(0) for Do.] 
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Table |—Observed Q-branch transition energies, », wavelengths, 4, 
and attenuation coefficients, a, HD room temperature data 


Fundamental 
lst Overtone 
2nd Overtone 
3rd Overtone 
4th Overtone 


Fundamental 
ist Overtone 
2nd Overtone 
3rd Overtone 
4th Overtone 


Fundamental 
ist Overtone 
2nd Overtone 
3rd Overtone 
4th Overtone 


(a) Measured coefficient for 500 atm, gas concentration = 5.6 x 10” molecules/cm’. 


(cm™’) 


4139 
8035 
11693 
15156 


2975 
5828 
8565 
11198 
13699 


3605 
7067 
10293 


H2 


nN 
(um) 


2.4159 
1.2445 
0.8552 
0.6598 


D, 


3.361 
1.716 
1.1675 
0.8930 
0.7300 


HD 


2.774 
1.415 
0.9715 


a(dB/km) ; 
Intensity 

(a) (b) (arb) 
4x 10° 800 — 
7100 14 — 
260 0.52 — 
13 0.026 — 
2100 4.2 a 
26 0.052 — 
3.1 6.2 x 10° — 
0.19 3.8x 10 — 
220 


(b) Derived coefficient for 1 atm, gas concentration = 1.1 X 10'* molecules/cm’. 
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Fig. 4—Correlation diagram of Raman and fundamental and overtone absorption 
spectra of H; in silica. Frequencies are measured with respect to the Q-branch frequency. 
The gas-phase H, rotational structure” and silica tetrahedral vibrations® are shown. 
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Fig. 5—Correlation diagram of Raman and overtone absorption spectra of D, in 
silica. Frequencies are measured with respect to the Q branch. The gas-phase D2 
rotational structure” and silica tetrahedral vibrations’ are shown. 


In contrast, the combination lines with the silica »,; and v3 modes 
show none of these effects; the separation from Q is the same in both 
the Hz and Dz spectra, and the intensities relative to Q and each other 
are independent of temperature. Even the fine details of Figs. 4 and 5 
support the above interpretation. The S(J) lines shift closer to Q for 
the higher-overtone absorptions, as expected, due to rotation-vibration 
coupling for a nonrigid rotor.® In contast, the lines due to combination 
with the silica modes show no such shift. 

Previous workers have observed many of the lines in the first- 
overtone spectrum reported here, but have made incorrect assign- 
ments, either assuming that all lines were due to combinations with 
silica modes,® which would indicate that the H. and D, did not rotate, 
or otherwise incorrectly identifying some of the transitions.*” In a 
Raman study of He and Dz in bulk silica glass, Hartwig and Vitko 
observed the S(J) lines and thus concluded that the dissolved species 
rotated in interstitial sites in the glass." 

The full set of spectra that we have recorded, of which only a portion 
is shown here, contains a wealth of detailed information concerning 
the nature of H, and D, dissolved in silica. In a subsequent publication 
we shall present the complete spectra, a complete set of assignments 
for all the absorption and Raman spectra, and full analyses of the 
spectra. Table II identifies the lines of most importance for lightwave 
communication, those due to Hz between 1.0 and 1.7 um. 
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Table 1!—Observed transitions between 
1.0 and 1.7 nm due to Hp dissolved ina 
silica optical fiber 


(um) Transition Spectral Origin 
1.696 Si(1) + v3 Fundamental 
1.642 S,(2) + v3 Fundamental 
1.590 S1(3) + v3 Fundamental 
1.550 Q: + 23 Fundamental 
1.2445 Qo First overtone 
1.197 S2(0) First overtone 
1.1682 S2(1) First overtone 
1.1455 S.(2) First overtone 
1.1320 Qe + ry First overtone 
1.0825 Qo + v3 First overtone 


The present work shows that Hz, even when dissolved close to 
saturation in silica, is still free to rotate. In contrast, when Ne is 
diffused into a silica fiber, only a single band is observed in the Raman 
spectrum, and contributions from molecular rotation are absent (al- 
though it is possible that these are not observed due to the small 
concentration of N>).!2 We also note confirmation of the fact that 
there is only a single gas molecule per site,’ since no lines due to 
double excitations are observed, despite the very high gas concentra- 
tion. 

For H, the strongest component of the first overtone, Qo, is very 
intense, about 7000 dB/km/5.6 x 10” molecules/cm’, but also very 
narrow. The absorption strength is about 70 dB/km/5.6 x 10” mole- 
cules/cm? at 1.30 wm. These values correspond to 14 dB/km at 1.2445 
pm, and 0.14 dB/km at 1.30 um, for the Hz concentration equivalent 
to 1 atm. Other reported values for the attenuation coefficient are 
considerably lower®**'* for the Q branch but are in good agreement?*® 
at 1.30 um. We believe this discrepancy in the peak attenuation 
coefficient of the Q branch is due to saturation that occurred in the 
other measurements of the intense Q-branch absorption. The absorp- 
tion in the 1.5 to 1.6-um portion of the spectrum is very large and 
increases rapidly with wavelength. The loss in this region is due to 
wings of the extremely intense fundamental absorption near 2.4 wm, 
which, like the overtones, is asymmetric and more intense toward 
shorter wavelengths. At 1.55 um the absorption due to the wing of the 
fundamental absorption seen in Fig. 1a is about 400 dB/km/5.6 x 107° 
molecule/cm? or 0.8 dB/km at 1 atm equivalent. Finally, we note that, 
even in the presence of Hz, there is a transmission window in silica 
from about 1.32 wm to about 1.45 wm, which might permit use of a 
hydrogen-contaminated fiber as a communications medium at a wave- 
length of approximately 1.4 wm. However, since this is the wavelength 
at which absorption in silica fibers due to OH contamination occurs, 
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such use would require the manufacture of fibers with negligible OH 
concentration. (No additional OH formation in silica due to Hz has 
been observed below about 100°C.)!4 

In summary, we have recorded and analyzed both absorption spectra 
up to the fourth overtone, and Raman spectra for fibers saturated with 
either Hz or Dz at 500 atm. The spectral lines have been completely 
assigned to vibration-rotation transitions of Hz (D2) and combination 
vibrational transitions with only the »; and v3 silica tetrahedral modes. 
The Hz (Dz) molecule is trapped singly and is free to rotate. 
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